Build visibility | Share expertise | Strengthen authority

  • CMiC Leaderboard - September 2026
  • Cooper Equipment Rentals - Leaderboard
  • Leaderboard First Aid Construction - Dentec
  • Keith leaderboard - Nov 2026
Data Lake Ingestion - Alice
October 8, 2025

Data Lake Ingestion Best Practices

Design a durable, low‑cost lake ingestion: domain folders, yyyy/mm/dd partitions, CSV vs Parquet, lifecycle rules, and COPY INTO.


TL;DR: Organize storage by domain, default to yyyy/mm/dd partitioning (ingest_date in bronze, event_date in silver), prefer Parquet for analytics and CSV only for interchange, enable lifecycle rules + versioning on day one, orchestrate with the Alice scheduler (retries & alerting), and use the COPY INTO pattern for incremental, idempotent loads.

  1. Folder Structure by Domain
  2. Date Partitioning (yyyy/mm/dd)
  3. CSV vs Parquet: When to Use Each
  4. Lifecycle Rules & Versioning
  5. Alice Scheduler: Retries, Timeouts, Alerting
  6. The COPY INTO Pattern & Incremental Loads
  7. Quick Checklist
  8. FAQs

Folder Structure by Domain (Predictable & Portable)

Keep your lake/lakehouse predictable across S3, GCS, or ADLS with a domain-first schema. Domains change slower than teams and map to how consumers actually think (sales, marketing, finance).

Keep reading this blog on alice.dev


Explore related articles you won’t want to miss:

Dealing with Dirty Data: How Alice Helps Normalize the Mess

Sync, Store, Serve: Azure Blob Storage Best Practices

  • BuildTech Week 2027 - Leaderboard
  • CMiC Leaderboard - September 2026
  • Cooper Equipment Rentals - Leaderboard
  • Geo Week 2027 - Leaderboard
  • World of Asphalt 2027 - Leaderboard
  • Canadian Concrete Expo 2027 - Leaderboard
  • Leaderboard First Aid Construction - Dentec
  • Keith leaderboard - Nov 2026