skinny.

Latest / DataTalks.Club

Build Your Own Data Pipeline - Andreas Kretz

We talked about:

Andreas’s background

Why data engineering is becoming more popular

Who to hire first – a data engineer or a data scientist?

How can I, as a data scientist, learn to build pipelines?

Don’t use too many tools

What is a data pipeline and why do we need it?

What is ingestion?

Can just one person build a data pipeline?

Approaches to building data pipelines for data scientists

Processing frameworks

Common setup for data pipelines — car price prediction

Productionizing the model with the help of a data pipeline

Scheduling

Orchestration

Start simple

Learning DevOps to implement data…

The skinny

The skinny isn't ready yet — notes appear once the transcript is processed.