Customer Insight Application Integration Hadoop/Spark
Environment: Hadoop YARN, Spark Core, Spark Streaming, Spark SQL, Scala, Python, Kafka, Hive, Sqoop, Amazon AWS, Elastic Search, Impala, Cassandra, Tableau, Talend, Oozie, Jenkins, Cloudera, Oracle 12c, Linux.
Skills: Java, Scala, Python, SQL, PL/SQL, Pig Latin, HiveQL, Unix, Java Script, Shell Scripting, HDFS, YARN, MapReduce, Hive, Pig, Impala, Sqoop, Flume, Spark, Kafka, Zookeeper, and Oozie, Storm, Spark, Kafka, Yarn and Zookeeper, Spark Streaming, Spark SQL, Spark MLib, Spring RDDs, AWS(EC2&EMR).
Description: The primary objective of this project is to integrate Hadoop (Big Data) with the Relationship Care Application to leverage the raw/processed data that the big data platform owns. It will provide an enriched customer experience by delivering customer insights, profile information and customer journey.
BIG DATA Playground (Big Data based on Docker & Kubernetes)
Environment: Hadoop YARN, Spark Core, Spark Streaming, Spark SQL, Scala, Kafka, Hive
Tools & Techniques used: Hadoop+HBase+Spark+Flink+Beam+ML stack, Docker & KUBERNETES, Kafka, MongoDB, AVRO, Parquet
Description: You will be creating a Batch/Streaming/ML/WebApp stack to locally test the jobs or submit to Yarn resource manager. Docker will be used to build the environment and Docker-compose will provision it with required components.