Spark architecture

Hi, I am a data engineer and currently working as in Digital Ads startup. I have 2 years of experience in setting up and running data pipelines. In my 2 years stint i worked on Hadoop, Apache Spark, Kafka, Storm on our AWS infrastructure. Please let me know what problems you are facing and I will definitely try and provide you with end-to end solution. Thanks

Hi, I am Amit. I have experience in Spark and Scala. I can design and develop the application using Spark. Please share the details of requirement. Looking forward to work on it. Thanks, Amit

Hi, I have 13+ years of experience on Windows and Linux servers especially built on VMware and Microsoft Hyper-V environments. I have working knowledge on various cloud services such as AWS, Azure and OpenStack. I am specialized on system administration, design, deployment and troubleshooting. I believe I would be an ideal candidate for this job. Regards, Shafiul

Hi there, I have been working in the Hadoop framework for over 3 years now. Hadoop has changed a lot from what it was in the initial years. From a mere low cost solution for large dataset storing via HDFS and analysis via MapReduce to a myriad of applications that is constantly evolving. The most used tools in the Hadoop ecosystem now a days are Spark (in memory proccessor that is slated to be the replacement of Mapreduce), Hive (the default database for the hadoop ecosystem tools), Impala, R, pig, Hbase etc. But the actual set of big data tools depends on the exact requirement of the project and the expected outcome. I am currently working in a few big data projects in my company and can surely help you to in your project as a Big Data Architect. In fact in the current project in my company, I am doing the exact same thing i.e. We are calculating Economic Capital i.e. Risk Capital of Insurance firms and using diffrent models for the calcualtion. The Sprak engine reads all the data from the Hive table, then does the Model calculation in its in memory spark engine and the results are written in another Hive table in the form of an OLAP Cube. Our visualization tool (Tableau) reads from this Hive Cube and dispalys the data to the end business user. We are also using Oozie to schedule workflows. Let me know if you have any questions. Thanks, Souvik

We bet you won't find more qualified technical team than us. Our technical Guys can face any technical questions from your side we challenge you won't get dissatisfied after working with us. you can check out our skills whenever you want. We guarantee that once we start working, you may charge us penalty in case our solution does not suits to your requirements thanks

I know spark memory tuning and benchmarking . spark streaming and SQL in depth to design architecture

I've so many experiences on construction and drawing. Also project management. We can talk much more about architecture, design, and engineering construction.

Having more than 3.2 years of experience in Spark & scala .I am successfully completed 4 projects in Hadoop .