136 Commits

Author SHA1 Message Date
891680bb05 linting 2025-07-26 19:44:11 +02:00
c184d062ef update pre-commit 2025-07-26 19:10:42 +02:00
66bcb3e2d3 towards detecting new deal 2025-07-26 19:09:06 +02:00
8d06b236b7 implement schema check and use automation instead of sensor 2025-07-26 17:53:12 +02:00
ac8759258d parse date column 2025-07-26 17:52:28 +02:00
8fe8d16ad9 data loading helper 2025-07-26 17:52:12 +02:00
cdd65b539a add patito dependency 2025-07-26 17:50:33 +02:00
433b52c0cb move parsing logic 2025-07-26 16:33:13 +02:00
fb2e90d47d rewrite parsing of deals 2025-07-26 16:28:26 +02:00
8a80adcd27 set timezone 2025-07-26 10:03:46 +02:00
62c25b32c8 rewrite to dagster as dg 2025-07-26 10:03:39 +02:00
ded11d9e60 store manual experiment 2025-07-26 09:26:32 +02:00
cb9a402610 vinyl repo is not yet using multi partition mapping 2025-07-26 09:25:10 +02:00
700b6d760d test write to stdout, multiple partitions 2025-07-26 09:24:22 +02:00
cc51ae2b6a delete unused partition mapping helper 2025-07-22 16:53:14 +02:00
6b5bda5cb2 improve sql 2025-07-22 13:42:31 +02:00
eb43acbb22 add patito dependency 2025-07-22 13:37:51 +02:00
91ca2a24a1 add existing partitions 2025-07-22 13:37:35 +02:00
be8fae969c fix paths to storage and logs 2025-07-22 12:03:03 +02:00
5b5e962773 linting 2025-07-22 11:35:18 +02:00
1a3c288001 tidy up 2025-07-21 10:56:46 +02:00
119583c730 mount all apps in container 2025-07-21 09:18:16 +02:00
fd73e1367c refactor to allow for multiple code locations 2025-07-20 19:49:30 +02:00
9b8cfabee5 add env vars 2025-06-07 15:35:27 +02:00
9578825b7e tidy up dagster config 2025-06-07 15:31:47 +02:00
20b03e3309 remove docker- prefix 2025-06-07 15:26:20 +02:00
796cda238e snapshot 2024-12-23 11:54:19 +01:00
e2964c95da persistent storage for database 2024-12-23 11:36:46 +01:00
f9c83d29b3 use polars and duckdb for lazy processing 2024-10-14 15:23:49 +02:00
1e0528bdfb collect unique works from deals dumps 2024-10-14 15:08:07 +02:00
6f2c4d1249 refactor plato logic 2024-10-14 13:10:24 +02:00
4b69bef47f add pre-commit 2024-10-14 12:52:11 +02:00
631ece7abc reindex plato columns 2024-10-14 12:50:09 +02:00
ebef914be6 linting 2024-10-14 12:49:32 +02:00
a7d8dfdbd3 update sounds fetch 2024-10-14 11:24:03 +02:00
0196f8bd27 initial commit 2024-10-14 09:58:24 +02:00