Realtime data capture and Processing using BigData Technologies(Kafka, MongoDB, Microservices,Cloud)

Project Description

The project is about capturing the data generated by a device in a realtime and processing the data to

derive some device metrics like device status and peripheral count.

The project defines a part of real time requirement from the client on capturing and processing telemetry

data from devices

The data should be captured using the push model from 'REST' services by a realtime messaging

system like 'Kafka' and processed by application services using the kafka libraries and derive the device

specific metrics and store in MongoDB.

Later on after deriving the metrics, same are to be retrived and represented using a dashboard.

The project should be executed as microservices by leveraging cloud infrastructure.

There exists a lot of scope for learning on-demand technologies like big data technologies, microservices

and cloud along with hand-on experience.

Project Outcome

The following are expected as the deliverables for this project

1. 'REST' service to mock up and pump realtime device data.

2. Application capturing data from 'REST' sources.

3. Application to process the real time data by applying the logic for device metrics and store in DB.

[login to view URL] to retrive the device metrics and represent in a dashboard

5.A dashboard to give a device centric view with various details specific to device configuration, device

status, etc.

Skills: Elasticsearch, Hadoop, node.js, NoSQL Couch & Mongo, Python

See more: kafka use cases, kafka projects, kafka real time processing, kafka real time example, spark kafka batch processing, kafka data pipeline, kafka real time examples, website activity tracking using kafka, saving grid data xml file using extjs, access database web data capture, data mapping oracle using informatica, web data capture robot php, automatic data entry table using javascript, data capture scripts, data weight machine using vbnet, send form data mysql database using php, ajax data grid system using php mysql, projects data processing using visual basic, market research data processing using quantum, Realtime data capture sharekhan tradetiger amibroker

About the Employer:
( 0 reviews ) Visakhapatnam, India

Project ID: #19000411

3 freelancers are bidding on average ₹5685 for this job


Hello there, We are a team of expert Big Data developers with more than 10 years of rich inductry experience & have succesfully delivered multiple projects in the past like a)recipe recommendation b)movie recom More

₹7777 INR in 3 days
(2 Reviews)

Hello, We are a group specializing in multiple cloud domains from Infra to networks, security, automation,PaaS and architecture Management. Would be happy to help you take the project further well withing defined ti More

₹7777 INR in 3 days
(0 Reviews)

I have good experience in implementing realtime event processing systems, using apache kafka, Cassandra, elastic search, etc and deploy as micro services on docker swarm cloud clusters

₹1500 INR in 5 days
(0 Reviews)