Find Jobs
Hire Freelancers

Data Engineeering

$250-750 USD

Closed
Posted almost 3 years ago

$250-750 USD

Paid on delivery
I need an expert in Pyspark. who can read the data from kafka topic and write it to hive tables applying required transformations.
Project ID: 30461593

About the project

18 proposals
Remote project
Active 3 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
18 freelancers are bidding on average $541 USD for this job
User Avatar
Hi, I saw that you need help with Python, PySpark, Apache Hadoop and Spark. I have 9 years of experience working on these frameworks. I believe I can help you with it. I would request you to have a look at my portfolio, customer feedbacks from my profile. If you find me worthy of doing your job. Please start the chat and lets discuss it. Regards Awais
$750 USD in 11 days
5.0 (7 reviews)
6.4
6.4
User Avatar
Hi there, I'm bidding on your project "Data Engineeering" Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for my reviews and portfolios. Thanks Note: This is not an automated bid and i'm ready to discuss your project right now. .. .
$750 USD in 5 days
5.0 (11 reviews)
6.1
6.1
User Avatar
Hi there,I'm biddin on your project "Data Engineeering" I have read your project description and i'm an expert in Python and machine learning therefore i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the project. Thanks. .. .
$750 USD in 3 days
4.8 (21 reviews)
6.2
6.2
User Avatar
Hello, I am very good at python data engineering. Please check my profile I hope to discuss more. Thank you. nemanja
$500 USD in 7 days
5.0 (19 reviews)
6.1
6.1
User Avatar
Hello! I am a python pyspark expert who has 10 years of experience and certifications. I would like to help you. Please let me know your choice.
$500 USD in 7 days
4.8 (14 reviews)
6.0
6.0
User Avatar
I read your project description carefully. I am bidding on your project because I am very much familiar with Data Engineering and tools like Kafka, Hadoop ECO system, Pyspark and Azkaban. I am an experienced Data Scientist and Machine Learning Engineer. Data Visualization, NLP, Deep learning, Artificial intelligence, machine learning, Data structures, and algorithms are my major fields. I finished specializations on Data Science, Machine learning, Deep neural Network, Convolution NN, Recurrent NN, Tuning Hyper Parameter .this project will well fit for me. I have won the 2nd runners up award in Sri Lanka Biggest Data Science Competition. I am very fluent with python and did a lot of data science and ml project. So I am familiar with these related libraries such as matplotlib, seaborn, pandas, numpy, sikit-learn, Keras, TensorFlow, spark etc .
$300 USD in 6 days
5.0 (74 reviews)
5.6
5.6
User Avatar
Hello, ‌Hope you doing well. We are a team of software developers. Delivered more than 100 projects based on ML, AI, application development, website and similar projects. Kindly message me in chat so that we can discuss about the project. ‌Regards Anchal
$500 USD in 15 days
4.9 (23 reviews)
5.1
5.1
User Avatar
hi, I recently completed a pyspark data pipeline project where data pulled from kafka producer, did training and prediction and written to elasticsearch and sent to kafka consumer topic using pyspark. You can see the review from my client. I did the complete ML model training and stream processing for prediction. Also I did it using latest version of Spark i.e. 3.1.1. I can do the same for hive table sink. I have five years of experience in building production data pipeline and ML/AI systems. Can we discuss in detail. Thank you.
$280 USD in 2 days
5.0 (10 reviews)
4.6
4.6
User Avatar
Hi, I am an experienced Data Engineer with a solid background in Spark. I have worked on many Big Data projects with Spark, Scala, Python, Cassandra, Kafka, Snowflake,... Let's have a call for more details about the project. Regards
$500 USD in 3 days
4.8 (3 reviews)
4.5
4.5
User Avatar
Hi Greetings! I have worked on similar projects working on streaming data using kafka clusters and big data applications using hadoop and spark. Used several datrabases including sql and nosql . Please inform further deails
$500 USD in 10 days
4.8 (9 reviews)
4.6
4.6
User Avatar
Hello This is Ajay singodia having expertise in python and experienced in MNCs as well worked with great teams now in my team we are 3 peoples. Thanks
$658 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
HI I have 1.5 years of experience in developing PySpark applications and I have 5 years of experience in developing Spark applications. If you elaborate your requirements I will be able to help you .If interested please ping me. Below approach can be used for solving your issue First need to read the data from corresponding Kafka topic into an RDD and then after doing the transformations using data frame operations we can load data into hive tables. The tables needs to be portioned based in the access pattern .
$500 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I have excellent skills in pyspark, kafka, python, ETL/ELT, DWH, bigdata, etc. With over 15+ years of exp in s/w development and over 10+ exp in data engineering. Provided a chance I should able to deliver the project with full enthusiasm. Hope to connect with you soon.
$800 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I have worked in python pyspark and hive query language. Worked in extracting data from json to pyspark dataframes and loading them into hive database.
$500 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I am a data engineer with 7 years of experience in ETL related projects. Let me know if I can help you with project. Also we can discuss the requirement over short meeting if you really interested. Thanks, Bala
$700 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I have been working on pyspark since 5 years now. I have recently developed Kafka- PySpark streaming application for my MNC client. I have dealt with variety of data sources having structued or unstructured format. Besides everything, I love churning data and absolutely love solving challenges the raw data can throw at you. Please let me know if we can complete this together. Best Regards, Sandy
$250 USD in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Overland Park, United States
0.0
0
Member since Oct 24, 2020

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.