Hire Data Engineers with Expertise in Python, Pandas, Scikit-learn, Agile, Windows, CentOS & More

Senior Data Engineer – Proficient in Python, Java, Scala, Big Data Technologies, AWS, Azure

Summary

Experienced Data Engineer with expertise in Python, Java, Scala, and a strong background in big data technologies, including Hadoop, Hive, Spark, and Kafka. Proficient in cloud platforms such as AWS and Azure. Skilled in machine learning, data analysis, and data cleaning using Pandas and Scikit-learn. Experienced in building data warehouses, designing and developing data processing pipelines, and working with large-scale distributed systems. Background in Banking, Finance, and E-commerce domains.

Work Experience

Leading Data Analytics Company (July 2021 – Present)

Senior Data Engineer

• Environment: JDK1.8, Python, Spark-SQL, PySpark, Oracle, Amazon S3, Cassandra, StreamSets, Control-M, Impala, Hue, Cyberark, Pandas, Numpy, WinSCP, Putty, Postman
• Project Description: Data processing from different upstream systems, ETL operations, and data visualization.
• Responsibilities: Data collection, ETL operations, data insights, tableau visualization, and more.

Global Top 20 IT Services Firm (Nov 2018 – May 2019)

Data Engineer

• Project Description: Unified Customer Preferences (UCP) for determining the best time to communicate with customers.
• Responsibilities: Data processing, transformation, communication with customers, and more.
• Environment: JDK 1.8, Python, Spark-SQL, PySpark, AWS S3, Glue, Athena, Putty, Cassandra.

Mid-Market IT Services Firm (April 2017 – July 2018)

Data Engineer

• Setting up and managing essential components such as Zookeeper server and Kafka broker.
• Reading data from input sources and streaming it into Kafka topics.
• Processing data streams using Spark Streaming, dividing them into batches (D-Stream).
• Managing and maintaining instances of Cassandra for data storage.
• Consuming data from Kafka topics and storing it in Cassandra databases.
• Creating reports using Zeppelin.
• Attending daily stand-up meetings to align with the team.

Leading Global Bank (September 2013 – December 2016)

Java Developer

Gained expertise in Java development and contributed to various projects during this tenure.

Education

Completed a B.Tech in Electronics and Communication from Biju Pattnaik University of Technology, Odisha in 2011, followed by a Master of Science (MS) degree in Machine Learning and Artificial Intelligence at Liverpool John Moore’s University, successfully completed in 2021.

Other

Achieved a Post-Graduation in Machine Learning and AI from IIIT-Bangalore in 2019, further enhancing expertise in artificial intelligence and machine learning. Additionally, obtained the Oracle certified Java (1.6) programmer (OCJP) certification, showcasing proficiency in Java programming. Proficient in Agile project methodology and experienced with operating systems including Windows, CentOS, and Ubuntu. Well-versed in Hadoop distribution, particularly Cloudera.