Commit Graph

73 Commits

Author SHA1 Message Date
38f8830521 tweaks on shuttle 2025-08-06 21:29:36 +02:00
28a195a256 tweaks 2025-08-04 20:05:20 +02:00
b26ba7aa35 tweaks 2025-08-04 19:09:32 +02:00
17ca8669ef parse stocks pages 2025-08-04 18:05:53 +02:00
f7318e85cd move utils 2025-08-04 18:02:00 +02:00
a3d931c1b3 update requirements files 2025-08-04 18:01:47 +02:00
b3121b4f32 daily parsing of raw weather 2025-08-04 16:27:36 +02:00
1bea1d2bb8 scan directly 2025-08-04 15:41:39 +02:00
4a44d4c3cd use batch weather fetcher 2025-08-04 15:15:53 +02:00
d6737cc327 raw nearby charging sites from proxy 2025-08-03 21:31:16 +02:00
7ac90a025a move html and json io managers 2025-08-03 21:31:06 +02:00
03f9979943 move html and json io managers 2025-08-03 21:30:12 +02:00
5392aceb26 fix works asset 2025-08-02 21:45:47 +02:00
d1999b904d update requirements 2025-08-02 21:36:21 +02:00
30f08d98b6 order good deals 2025-08-02 21:14:43 +02:00
076dda0ee7 custom run launcher for custom docker image 2025-08-02 21:13:41 +02:00
94dcdd04a7 custom run launcher for custom docker image 2025-08-02 21:06:42 +02:00
b1d115fe80 scaffold nearby charging sites logic 2025-08-02 10:51:01 +02:00
6bd5e6d571 reduce sensor frequency 2025-08-02 10:50:13 +02:00
b5f7952c07 dynamic partition names 2025-08-02 10:49:56 +02:00
1d7df06dcf handle missing partitions 2025-07-30 21:53:10 +02:00
02db619c6d store and register scrape 2025-07-29 17:22:42 +02:00
0539dd9f7e move partition definition 2025-07-29 17:21:58 +02:00
09e6583579 experiments with materialization 2025-07-29 17:06:11 +02:00
4c59dcb8ac demo of delta lake with forked io manager 2025-07-29 15:37:37 +02:00
f59b30b9ea shared config 2025-07-29 11:38:41 +02:00
586a4ae904 use playwright to scrape stocks page 2025-07-29 11:31:26 +02:00
8aa943b0bf html io manager 2025-07-29 11:30:17 +02:00
f785a7d3b1 correct name 2025-07-29 11:29:18 +02:00
d74823960f shared base for file IO manager 2025-07-29 11:28:44 +02:00
80c0452cde linting 2025-07-29 11:27:22 +02:00
12bf5b1b87 update dependencies 2025-07-29 11:26:03 +02:00
b386a375b5 selenium snapshot 2025-07-28 22:33:31 +02:00
8329d7ed68 scaffold sensor triggered html download 2025-07-28 19:38:37 +02:00
25cccdb501 towards batch download 2025-07-28 19:37:52 +02:00
78de29e930 add tag to job 2025-07-27 18:27:06 +02:00
e4dddd0400 put vinyl assets under key 2025-07-27 18:17:56 +02:00
8e8b6190b9 remove test code 2025-07-27 18:16:36 +02:00
1e9078fed5 add asset key prefix 2025-07-27 18:06:47 +02:00
e5cb15d1ba organize weather data by retrieval time 2025-07-27 17:52:31 +02:00
662bc239f9 polars io manager ignores partition_by metadata 2025-07-27 17:51:52 +02:00
f3c2f9ec11 error handling and remove from cache 2025-07-27 16:57:59 +02:00
a8b8142402 combine sounds and plato deals, use jinja for email 2025-07-27 16:43:08 +02:00
95b6513da9 tidy up 2025-07-27 15:23:54 +02:00
3aa6790f29 tidy up 2025-07-27 15:23:35 +02:00
05c2b33e25 it's not working 2025-07-27 15:23:25 +02:00
d1b66be636 attempt to overwrite run launch container kwargs 2025-07-27 15:22:12 +02:00
131912b70d first version of weather loader 2025-07-27 14:54:19 +02:00
44aee9c54d store requirements files 2025-07-26 22:17:51 +02:00
fd1603e309 scaffold other code locations 2025-07-26 22:17:22 +02:00