This software is
- quite old (especially regarding TypeScript/JS coding conventions)
- not well suited for larger collections (no search, no tagging etc.)
- barely updated (dependencies are out of date, some of them no longer supported etc)
- I think this server doesn't work with newer versions of the
website-scraper node module, so it's still on an older version
While it still works (I've had it running for 4+ years), it feels like it's time for an overhaul. The following could be good steps:
- Creating a better frontend (the code in
browser.js works, but is a mess)
- Creating a docker (compose) configuration that just works to simplify installation
- Maybe switch to a database instead of just storing everything in a JSON file (not sure I'll do this, as it complicates setup etc.)
- Updating the downloader code to use newer website-scraper versions
2 & 4 should be possible without major rewrites, the others are somewhat more involved.
Maybe it also makes sense to rewrite the server in Go because I like it better, but I'm not sure if there's a good website-scraper-like module for Go. I actually tried writing something like that that started a chromium browser and used SingleFile to download pages, but it didn't work that well.
I'm not sure when (or if) I'll work on this.
This software is
website-scrapernode module, so it's still on an older versionWhile it still works (I've had it running for 4+ years), it feels like it's time for an overhaul. The following could be good steps:
browser.jsworks, but is a mess)2 & 4 should be possible without major rewrites, the others are somewhat more involved.
Maybe it also makes sense to rewrite the server in Go because I like it better, but I'm not sure if there's a good
website-scraper-like module for Go. I actually tried writing something like that that started a chromium browser and used SingleFile to download pages, but it didn't work that well.I'm not sure when (or if) I'll work on this.