Commit Graph

112 Commits

Author SHA1 Message Date
8329d7ed68 scaffold sensor triggered html download 2025-07-28 19:38:37 +02:00
25cccdb501 towards batch download 2025-07-28 19:37:52 +02:00
78de29e930 add tag to job 2025-07-27 18:27:06 +02:00
e4dddd0400 put vinyl assets under key 2025-07-27 18:17:56 +02:00
8e8b6190b9 remove test code 2025-07-27 18:16:36 +02:00
62efbb4a41 adjust concurrency 2025-07-27 18:12:48 +02:00
1e9078fed5 add asset key prefix 2025-07-27 18:06:47 +02:00
e5cb15d1ba organize weather data by retrieval time 2025-07-27 17:52:31 +02:00
662bc239f9 polars io manager ignores partition_by metadata 2025-07-27 17:51:52 +02:00
f3c2f9ec11 error handling and remove from cache 2025-07-27 16:57:59 +02:00
a8b8142402 combine sounds and plato deals, use jinja for email 2025-07-27 16:43:08 +02:00
f254afbdbb use envsubst to configure stuff in dagster.yaml 2025-07-27 15:41:10 +02:00
e756f92d54 add some dependencies 2025-07-27 15:24:04 +02:00
95b6513da9 tidy up 2025-07-27 15:23:54 +02:00
3aa6790f29 tidy up 2025-07-27 15:23:35 +02:00
05c2b33e25 it's not working 2025-07-27 15:23:25 +02:00
d1b66be636 attempt to overwrite run launch container kwargs 2025-07-27 15:22:12 +02:00
131912b70d first version of weather loader 2025-07-27 14:54:19 +02:00
d3d3911609 add mailserver settings to daemon 2025-07-27 14:50:14 +02:00
44aee9c54d store requirements files 2025-07-26 22:17:51 +02:00
fd1603e309 scaffold other code locations 2025-07-26 22:17:22 +02:00
ea4bee7186 prettier emails 2025-07-26 21:28:34 +02:00
5c188c2190 pass timezone 2025-07-26 21:22:36 +02:00
cdf5055e4f send email 2025-07-26 21:15:59 +02:00
d55cb8fb17 correct 2025-07-26 19:53:13 +02:00
329a50a5d6 find good deals 2025-07-26 19:44:39 +02:00
891680bb05 linting 2025-07-26 19:44:11 +02:00
c184d062ef update pre-commit 2025-07-26 19:10:42 +02:00
66bcb3e2d3 towards detecting new deal 2025-07-26 19:09:06 +02:00
8d06b236b7 implement schema check and use automation instead of sensor 2025-07-26 17:53:12 +02:00
ac8759258d parse date column 2025-07-26 17:52:28 +02:00
8fe8d16ad9 data loading helper 2025-07-26 17:52:12 +02:00
cdd65b539a add patito dependency 2025-07-26 17:50:33 +02:00
433b52c0cb move parsing logic 2025-07-26 16:33:13 +02:00
fb2e90d47d rewrite parsing of deals 2025-07-26 16:28:26 +02:00
8a80adcd27 set timezone 2025-07-26 10:03:46 +02:00
62c25b32c8 rewrite to dagster as dg 2025-07-26 10:03:39 +02:00
ded11d9e60 store manual experiment 2025-07-26 09:26:32 +02:00
cb9a402610 vinyl repo is not yet using multi partition mapping 2025-07-26 09:25:10 +02:00
700b6d760d test write to stdout, multiple partitions 2025-07-26 09:24:22 +02:00
cc51ae2b6a delete unused partition mapping helper 2025-07-22 16:53:14 +02:00
6b5bda5cb2 improve sql 2025-07-22 13:42:31 +02:00
eb43acbb22 add patito dependency 2025-07-22 13:37:51 +02:00
91ca2a24a1 add existing partitions 2025-07-22 13:37:35 +02:00
be8fae969c fix paths to storage and logs 2025-07-22 12:03:03 +02:00
5b5e962773 linting 2025-07-22 11:35:18 +02:00
1a3c288001 tidy up 2025-07-21 10:56:46 +02:00
119583c730 mount all apps in container 2025-07-21 09:18:16 +02:00
fd73e1367c refactor to allow for multiple code locations 2025-07-20 19:49:30 +02:00
9b8cfabee5 add env vars 2025-06-07 15:35:27 +02:00