implement bulk download API endpoints, polling, and file downloader#245
Open
implement bulk download API endpoints, polling, and file downloader#245
Conversation
This: - add `get_vessel_insights()` for interacting with the vessels insights API - add `with_gfw_json_fixture()` helper for loading fixture JSON files in tests
This also: - export `get_vessel_insights` and update NAMESPACE - improve validation, error messages, and date parsing - default API key to `gfw_auth()` - refactor request body construction for Insights API - update examples, fixtures, and tests to match new interface - reformat JSON fixtures for consistency and clarity
…tion tests This: - add `print_request` flag to optionally print the HTTP request for debugging - fix typo in missing API token error message - expand unit tests to cover NULL, NA, empty string, and envvar-based key errors - improve documentation and examples for the new debug option
…bble This: - improve documentation with detailed @description, @details, and @examples - normalize Insights API response into a single-row tibble with list-columns - update unit tests to assert tibble structure and exact response mapping
This: - addresses and closes #162 - rename `get_vessel_insights()` to `gfw_vessel_insights()` - update roxygen examples to use new function name - regenerate `NAMESPACE` to export `gfw_vessel_insights` instead of `get_vessel_insights` - update all testthat references and expectations to use `gfw_vessel_insights`
…st coverage This: - expand insight types to include `"VESSEL-IDENTITY-FLAG-CHANGES"` and `"VESSEL-IDENTITY-MOU-LIST"` - update `gfw_vessel_insights()` documentation to reflect additional insight types - add examples demonstrating retrieval of flag change insights and Tokyo/Paris MOU list insights - update JSON test fixtures to include flag change history and MOU list presence - extend unit tests structural assertions for period, gap, coverage, fishing, and vessel identity
…dpoints This: - add `@details` sections linking to official Global Fishing Watch API documentation - include data caveat references for 4Wings, Events, Regions, and Vessels APIs
This: - expand `README.Rmd` notes to include links to Datasets, Data Caveats, and Terms of Use - add contextual documentation references for `Vessels`, `Events`, `4Wings`, and `Regions` sections
This: - add standardized `"Note"` sections linking to `Datasets`, `Data Caveats`, and `Terms of Use` - include data caveat references across `fishing activity`, `vessel presence`, `identity`, and `SAR` vignettes
This: - add JSON fixtures for bulk report create request and API response payloads - add validation tests for required parameters (`name`, `dataset`, `format`, `filters`, etc.) - verify error handling for missing, empty, and invalid input values - add unit test validating bulk report creation workflow
This: - add `gfw_get_bulk_report_by_id()` to retrieve bulk report metadata and status - fix request body typo (`dateset` → `dataset`) in bulk report creation - standardize internal section comments and docstrings - add tests covering id validation, token validation, and successful API responses
…nd tests This: - use rlang string validation helpers - switch to `httr2::req_auth_bearer_token()` - simplify request body construction - remove redundant response normalization - hide internal docs with `@noRd` - make tests less brittle to API changes
This: - add `gfw_get_bulk_report_file_download_url()` to get signed download URLs for bulk report files - support `DATA`, `README`, and `GEOM` bulk report file types with validation - add JSON fixtures for signed file download URL API response payloads - add unit test validating get bulk report signed file download URL workflow
This: - add `gfw_get_all_bulk_reports()` to retrieve a list of metadata and status for bulk reports - implement input validation for `limit`, `offset`, `sort`, and `status` - add unit tests for parameter validation and API request/response workflows - include JSON fixture for bulk report list request parameters
…tion This: - Switch to `httr2::req_auth_bearer_token()` and `rlang` validation helpers - Use `tibble::as_tibble_row()` for cleaner single-object API responses
This: - Update roxygen documentation to define tibble schemas for: - `gfw_create_bulk_report()`, - `gfw_get_bulk_report_by_id()`, - `gfw_get_bulk_report_file_download_url()`, - and `gfw_get_all_bulk_reports()`
…e mapping This: - Add roxygen `@examples` for: - `gfw_create_bulk_report()`, - `gfw_get_bulk_report_by_id()`, - `gfw_get_bulk_report_file_download_url()`, - and `gfw_get_all_bulk_reports()` - Update response normalization to convert JSON `nulls` to `NA`
…reation This: - Add `region` parameter and ensure it is a list containing valid `dataset` and `id` keys - Add `geojson` parameter and ensure it is a sf/sfc/sfg, JSON strings, and named lists - Set default parameters for `dataset` and `format` - Add unit tests for `region` and `geojson` parameter validation
This: - Implement `gfw_wait_for_bulk_report()` to handle asynchronous report generation - Add roxygen2 documentation including return schemas and usage examples - Add unit tests covering terminal statuses, polling iterations, and `max_runtime` expiration
This: - Add `gfw_download_bulk_report_file()` to download bulk report artifacts (DATA, README, GEOM) - Support configurable `data_dir` with fallback to `GFW_DATA_DIR` and `R_user_dir` cache - Add roxygen2 documentation and usage examples - Add `with_gfw_mocked_data_dir()` test helper - Extend `with_gfw_mocked_envvar()` to include `GFW_DATA_DIR` - Add unit tests for URL validation, and file download request/response workflows - Add `DATA`, `README`, and `GEOM` fixtures for report file download validation
lykmapipo
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This:
gfw_wait_for_bulk_report()polling utility to handle asynchronous report generationgfw_download_bulk_report_file()with configurable data directories