cronjob: updates required for GRASS 8.5.0 release#1692
Conversation
This PR extends OSGeo#1689 and updates all cronjobs as follows: - correctly define GRASS versions (as to be modified after GRASS 8.5.0 release) - update of comments "preview" (8.6) and "current" (8.5): - fetch HTML manual pages from GitHub (created via GitHub "documentation" action) - remove internal creation of HTML manual - incorporated `fetch_unpack_manual_GHA.sh` functionality Removed `fetch_unpack_manual_GHA.sh` as no longer needed and updated cronjob list accordingly.
| RUN_ID=$(gh run list \ | ||
| --branch main \ | ||
| --branch "$MYBRANCH" \ | ||
| --repo "$OWNER/$REPO" \ | ||
| --workflow "$WORKFLOW_NAME" \ | ||
| --status success \ | ||
| --limit 1 \ | ||
| --json databaseId \ | ||
| --jq '.[0].databaseId') |
There was a problem hiding this comment.
This doesn't go fetch the latest commit-wise I think. It's only the latest ran. So if we do a rerun of an older failed job, like 3 months later, it'll go get that artifact, if my understanding is right. And that's not what we want.
There was a problem hiding this comment.
Agreed. So we need a more sophisticated use of gh here.
There was a problem hiding this comment.
Idea: name the documentation artifact with YYYYMMDD-<branch>-mkdocs.zip using the commit date (rather than the workflow date). Then download the latest file to the server etc. That would be done in the "documentation" workflow.
| (cd $TARGETHTMLDIRSTABLE ; rm -rf barscales colortables icons northarrows) | ||
| # clone manual pages | ||
| cp -rp $TARGETHTMLDIR/* $TARGETHTMLDIRSTABLE/ | ||
| rm -rf $TARGETHTMLDIRSTABLE |
There was a problem hiding this comment.
So that means that temporarily, the docs will be unavailable if they are deleted before being rebuilt? (If the process doesn't finish its even worse).
What about having multiple "staging" folders, not served directly, that are like grass85-20260509205104 or something unique enough and the rebuild/unzipping of artifact is done in there, and when done, a symlink swaps what folder is served, atomically. Keep last 2-3 versions on the server. You could swap the symlink to a previous version if something goes wrong.
For not having these folders served but available to be served as symlink, there might be some friction with the configuration of some web servers, which might try to prevent you from serving something outside what is already publicly served.
There was a problem hiding this comment.
Generally I agree. And would welcome implementation support.
| # clone manual pages | ||
| cp -rp $TARGETHTMLDIR/* $TARGETHTMLDIRSTABLE/ | ||
| rm -rf $TARGETHTMLDIRSTABLE | ||
| rsync -a $TARGETHTMLDIR/ $TARGETHTMLDIRSTABLE/ |
There was a problem hiding this comment.
Is there a flag needed to propagate the deletions (if the line above is changed according to my other observation)?
There was a problem hiding this comment.
Due to the hard cleanup a line below we here left out the (potentially dangerous) delete flag.
| rm -rf $TARGETHTMLDIRDEVEL | ||
| rsync -a $TARGETHTMLDIR/ $TARGETHTMLDIRDEVEL/ |
There was a problem hiding this comment.
Similar observations as for stable versions
|
Note: since the test implementation of this PR on the server (as there is no "dry-run" environment), I discovered that this sitemap is actually not existing on the server: Means: there is a naming confusion. See vs https://github.com/OSGeo/grass/blob/5bd94dabee99556bf4956e0fd933617a61676085/.github/workflows/documentation.yml#L250 Which name shall be used? Unless this is fixed, the entire "devel" manual is no longer shown when using search engines like Google etc: |
|
Keep in mind, from what I remember of playing around last year, the generated sitemap is not perfect. and collide, meaning we can't configure correctly, as that last line will win. Either the canonical rel links are correct, or the sitemap is correct. |
How about calling it |
No, it’s the html_baseurl that is assigned twice |
|
Why is that relevant? Sorry, I don't get it. |
2 participants
This PR extends #1689 and updates all cronjobs as follows:
Overall:
Scripts "preview" (8.6) and "current" (8.5):
fetch_unpack_manual_GHA.shfunctionalityadded/synced sphinx support(removed in b90e6d3)Script
gh_cli_download_artifact.sh:Removed
fetch_unpack_manual_GHA.shas no longer needed and updated cronjob list accordingly.On the server grass.osgeo.org: "grass-stable/" and "grass-devel/" are true directories (and neither Apache redirects nor links on disk). They are sync'ed in above scripts using
rsyncfrom versioned source directories.