DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Repartition vs Coalesce vs partitionBy on Azure Databricks: Controlling Output Files When Writing Delta Tables
karanpulyala profile

Repartition vs Coalesce vs partitionBy on Azure Databricks: Controlling Output Files When Writing Delta Tables

#databricks #pyspark #spark #dataengineering
Comments
6 min read
Stripe reconciliation: carrying September into October
akarazhev profile

Stripe reconciliation: carrying September into October

#dataengineering #sql #fintech #stripe
Comments
4 min read
From Google- and Meta-Like CSVs to a Trusted Schema: Building Fail-Closed Python Adapters
rupak_kulkarni_08b45c5846 profile

From Google- and Meta-Like CSVs to a Trusted Schema: Building Fail-Closed Python Adapters

#python #dataengineering #etl #testing
Comments
6 min read
Designing a movie ticket booking system: the low-level design interview, explained
gowthampotureddi profile

Designing a movie ticket booking system: the low-level design interview, explained

#walkin #interview #designsystem #dataengineering
Comments
16 min read
n8n Bridges Workflow Automation and API Management, With Important Trade-offs
aidatalegends profile

n8n Bridges Workflow Automation and API Management, With Important Trade-offs

#aiagents #apimanagement #dataengineering #n8n
Comments
3 min read
Databricks Lakehouse Platform for Real-Time, Agentic Data and AI Workloads
aidatalegends profile

Databricks Lakehouse Platform for Real-Time, Agentic Data and AI Workloads

#aiagents #dataengineering #databricks #lakehouse
Comments
3 min read
Data Lake vs Data Warehouse vs Data Lakehouse: Data Engineering Simplified
thejoud1997 profile

Data Lake vs Data Warehouse vs Data Lakehouse: Data Engineering Simplified

#dataengineering #systemdesign #database #abotwrotethis
1
Comments
2 min read
Databricks Runtime Introduces NEAREST BY: Native SQL Vector Search at Scale
aidatalegends profile

Databricks Runtime Introduces NEAREST BY: Native SQL Vector Search at Scale

#dataengineering #databricks #lakehouse #sql
Comments
4 min read
Querying Parquet on S3 with DuckDB: secrets, speed and the settings that matter
mohammed_arshadansari_f2 profile

Querying Parquet on S3 with DuckDB: secrets, speed and the settings that matter

#dataengineering #database #sql #tutorial
Comments
3 min read
Designing an ATM: the object-oriented design interview, explained
gowthampotureddi profile

Designing an ATM: the object-oriented design interview, explained

#interview #walkin #designsystem #dataengineering
Comments
15 min read
A watch alert said 2 new reviews. One of them never changes.
fetchsmith profile

A watch alert said 2 new reviews. One of them never changes.

#webscraping #dataengineering #api #javascript
Comments
4 min read
Tiered Storage in Apache Kafka — Hot Data on Disk, Old Data in S3 (and the Same Trick That Cuts a Databricks Bill)
charanrachumalla profile

Tiered Storage in Apache Kafka — Hot Data on Disk, Old Data in S3 (and the Same Trick That Cuts a Databricks Bill)

#kafka #dataengineering #databricks #kubernetes
1
Comments 1
8 min read
Auto CDF Moved the CDC Tax From Writes to Reads: Delta CDF vs Auto CDF vs Spark 4.2 CHANGES
firfircelik profile

Auto CDF Moved the CDC Tax From Writes to Reads: Delta CDF vs Auto CDF vs Spark 4.2 CHANGES

#dataengineering #databricks #spark #deltalake
1
Comments 1
9 min read
Why I Failed the DP-700 Exam, and How I Passed It
revos_ai profile

Why I Failed the DP-700 Exam, and How I Passed It

#dataengineering #microsoft #career #learning
2
Comments
9 min read
Designing Splitwise: the low-level design interview, explained
gowthampotureddi profile

Designing Splitwise: the low-level design interview, explained

#walkin #interview #dataengineering #designsystem
Comments
15 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.