Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Contributing to WebARKitLib

Thanks for helping improve WebARKit! This is the C/C++ source of the tracker; most
changes are consumed by the [webarkit-testing](https://github.com/webarkit/webarkit-testing)
superproject as a submodule.

## Workflow

- **Branch from `dev`** and open your PR **against `dev`** (not `main`).
- Reference the related issue in the PR description.
- **Sign your commits** (`git commit -S …`).

## Commit messages — Conventional Commits

All commit messages **must** follow
[Conventional Commits](https://www.conventionalcommits.org/):

```
<type>(<optional scope>): <short description>
```

Common types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`, `perf`. Examples:

```
feat(tracker): detect features on a downsampled frame
fix(#55): populate pose3d so getPoseMatrixCV() isn't zero
docs: document the pose pipeline
refactor(#50): remove dead computePose/invertPose
```

## What you may change

- Active WebAR work lives under **`WebARKit/`** (the OCVT planar tracker).
- **Do not** modify the vendored **old ARToolKit5** sources under `lib/SRC/**` or
`include/AR/**` — those are upstream utilities used by
[jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) /
[ARnft](https://github.com/webarkit/ARnft) and are kept untouched.

## Cross-repo changes (the PR pair)

> **Applies to `WebARKit/` (this folder) ↔ the `webarkit-testing` repo.** This flow is
> for changes to the **`WebARKit/`** OCVT tracker, which the **webarkit-testing**
> superproject compiles to WASM. Changes to the vendored `lib/SRC/**` / `include/AR/**`
> (old ARToolKit5) do **not** use this flow — they reach the web through
> [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT), a different consumer.

A change to `WebARKit/` that affects the WASM build is consumed by webarkit-testing via
the submodule. Coordinate both repos:

1. Branch from `dev` in **both** WebARKitLib and webarkit-testing.
2. Make the C++ change here; in webarkit-testing bump the submodule pointer and
rebuild (`npm run build-docker` → `build-es6`).
3. Open a **PR pair** (WebARKitLib → `dev`, webarkit-testing → `dev`); cross-link them.
4. After the WebARKitLib PR merges, re-point the submodule to the merged `dev` tip.

## Building & tests

- The library is normally built **from webarkit-testing** (WASM) — see its README.
`WebARKit/CMakeLists.txt` can also build it as a static lib for **WASM** or **native
Linux** (selected via `EMSCRIPTEN_COMP`), pulling OpenCV from
[webarkit/opencv-em](https://github.com/webarkit/opencv-em).
- **Unit tests** (GoogleTest) live in `tests/` and run in CI on every PR:

```bash
cmake -S tests -B tests/build && cmake --build tests/build && ctest --test-dir tests/build
```

If you change a config constant or the version, update the matching assertions in
`tests/webarkit_test.cc`.

## License

By contributing, you agree that your contributions are licensed under **LGPL-3.0**.
97 changes: 90 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,98 @@

![github releases](https://flat.badgen.net/github/release/webarkit/WebARKitLib)
![github stars](https://flat.badgen.net/github/stars/webarkit/WebARKitLib)
![github forks](https://flat.badgen.net/github/forks/webarkit/WebARKitLib)[![Test](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml/badge.svg)](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml)

# WebARKitLib

The C/C++ source code of WebARKit, from Artoolkit5 and extended.
The C/C++ source of **WebARKit** — a browser-side Augmented Reality tracker. The
active code (`WebARKit/`) is an emscripten-friendly port of the **OCVT** planar
image-tracking design from [ArtoolkitX](https://github.com/artoolkitx/artoolkitx),
built on OpenCV. It is normally compiled to WebAssembly and consumed from the
[webarkit-testing](https://github.com/webarkit/webarkit-testing) superproject.

> **Two lineages live here, keep them straight:**
> - **`WebARKit/`** — the WebAR OCVT planar tracker (the design reference is *ArtoolkitX*). This is the code under active development.
> - **`lib/SRC/**`, `include/AR/**`** — vendored **old ARToolKit5** utility sources/headers, kept only because [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) needs them (and [ARnft](https://github.com/webarkit/ARnft) in turn consumes jsartoolkitNFT as an npm module). Treated as untouched upstream. (Updated to Eigen 3.4.0 / emsdk 3.1.26.)

## Code layout (`WebARKit/`)

| Component | Role |
|---|---|
| `WebARKitManager` | top-level façade — owns the tracker, drives init/process, exposes the output (homography, pose, GL-view, projection) |
| `WebARKitTrackers/WebARKitOpticalTracking/WebARKitTracker` | the OCVT tracker core: feature detect/match → homography → optical-flow + template-match tracking |
| `…/TrackingPointSelector` + `…/TrackedPoint` | bin-based selection of template tracking points (ArtoolkitX-derived) |
| `…/WebARKitHomographyInfo` + `WebARKitUtils.h` | RANSAC homography estimation/validation (`getHomographyInliers`) |
| `…/WebARKitConfig` | tuning constants (feature counts, match ratios, pyramid/template sizes, version) |
| `…/TrackerVisualization` | optional debug-overlay scaffolding (inert until wired) |
| `WebARKitCamera` | camera intrinsics from a diagonal-FOV estimate |
| `WebARKitGL` | CV→GL matrix conversion (`arglCameraViewRHf`) + GL projection |
| `WebARKitPattern` | the reference marker (`WebARKitPattern`) and per-frame pose state (`WebARKitPatternTrackingInfo`) |
| `WebARKitLog` | logging |

## Pose pipeline (one glance)

```
solvePnP (cameraPoseFromPoints) // OpenCV-convention camera pose
-> getTrackablePose / updateTrackable // CV->GL handedness fix, D*R*D (see #42)
-> arglCameraViewRHf // right-handed GL modelview
-> matrixGL_RH // what the examples attach content to
```

Pose getters: **`getPoseMatrixCV()`** (raw 4×4 OpenCV pose) and **`getPoseMatrixGL()`**
(the GL/right-handed pose); projection via `getCameraProjectionMatrix()`. The full
derivation is in
[`docs/design-projection-and-pose-artoolkitx-alignment.md`](https://github.com/webarkit/webarkit-testing/blob/dev/docs/design-projection-and-pose-artoolkitx-alignment.md)
(webarkit-testing).

## Building

This repo isn't built as a single unit — each consumer compiles the **subset it
needs**:

- **`WebARKit/` (the OCVT tracker)** is built by the
[webarkit-testing](https://github.com/webarkit/webarkit-testing) superproject, which
compiles it to WebAssembly with emscripten (emsdk **3.1.26**) inside Docker and wires
it to JS through `emscripten/WebARKitJS.{cpp,h}` + `bindings.cpp` (which live in
webarkit-testing). webarkit-testing currently drives that compile with the Node script
`tools/makem.js` (`npm run build-docker` → `build-es6`); migrating to the CMake config
(`WebARKit/CMakeLists.txt`) is planned.
- **The vendored `lib/SRC/**` + `include/AR/**` (old ARToolKit5)** is built by
[jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) as part of its build —
not by webarkit-testing. ([ARnft](https://github.com/webarkit/ARnft) consumes
jsartoolkitNFT as an npm module and doesn't build the emscripten code itself.)

`WebARKit/CMakeLists.txt` can **already build WebARKitLib as a static library**
today — for both **WASM** (emscripten) and **native Linux**, selected via the
`EMSCRIPTEN_COMP` flag. It depends on a prebuilt **OpenCV** from
[webarkit/opencv-em](https://github.com/webarkit/opencv-em) (the emscripten
`opencv-js-…-emcc` build for WASM, or the native `opencv-…` build for Linux),
fetched automatically via CMake `FetchContent`. The "planned" part above is
webarkit-testing switching its WASM build over to this CMake config (away from
`tools/makem.js`). The same config also drives the unit tests (see below).

## Tests

C++ unit tests (GoogleTest) live in [`tests/`](tests/) (`webarkit_test.cc`,
`CMakeLists.txt`, `pinball.jpg`) and run in CI via
[`.github/workflows/test.yml`](https://github.com/webarkit/WebARKitLib/actions/workflows/test.yml).
Build them standalone with CMake:

```bash
cmake -S tests -B tests/build && cmake --build tests/build && ctest --test-dir tests/build
```

## Contributing

Please read [`./CONTRIBUTING.md`](./CONTRIBUTING.md) for the full guidelines. In short:

Updated to [Eigen](https://eigen.tuxfamily.org) 3.4.0 and Emscripten emsdk 3.1.26.
- Branch from `dev`; PR back to `dev`; sign your commits and reference the issue.
- **Commit messages must follow [Conventional Commits](https://www.conventionalcommits.org/)**
(e.g. `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`).
- Library changes are usually paired with a build/bump in webarkit-testing — see that
repo's "cross-repo PR pair" flow.
- Don't modify the vendored `lib/SRC/**` / `include/AR/**` (old ARToolKit5) as part of
WebAR work.

## Features
## License

The library containes the following features:
**Artoolkit5** C/C++ code used by [jsartoolkitNFT](https://github.com/webarkit/jsartoolkitNFT) and [ARnft](https://github.com/webarkit/ARnft) (see lib and include folders). Note that is not the full Artoolkit5 library, but only the code needed by jsartoolkitNFT and ARnft. The code is updated to Eigen 3.4.0 and can be compiled with Emscripten emsdk 3.1.26.
**WebARKit** C++ code, which is our experimental code for WebAR. It is completetly independent from Artoolkit5 and can be used as a standalone library. It can be compiled with Emscripten emsdk 3.1.26.
LGPL-3.0 (see `LICENSE.txt`).
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,19 @@ extern const cv::Size blurSize(3, 3);
extern const double ransac_thresh = 2.5f;
extern cv::RNG rng( 0xFFFFFFFF );
extern const double m_pi = 3.14159265358979323846;
extern const std::string WEBARKIT_HEADER_VERSION_STRING = "1.0.0";
extern const std::string WEBARKIT_HEADER_VERSION_STRING = "0.8.0";
/*@
The MAJOR version number defines non-backwards compatible
changes in the ARToolKit API. Range: [0-99].
*/
extern const int WEBARKIT_HEADER_VERSION_MAJOR = 1;
extern const int WEBARKIT_HEADER_VERSION_MAJOR = 0;

/*@
The MINOR version number defines additions to the ARToolKit
API, or (occsasionally) other significant backwards-compatible
changes in runtime functionality. Range: [0-99].
*/
extern const int WEBARKIT_HEADER_VERSION_MINOR = 0;
extern const int WEBARKIT_HEADER_VERSION_MINOR = 8;

/*@
The TINY version number defines bug-fixes to existing
Expand Down
4 changes: 2 additions & 2 deletions tests/webarkit_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ TEST(WebARKitConfigTest, TestConfigValues) {
EXPECT_EQ(maxLevel, 3);
EXPECT_EQ(featureDetectPyramidLevel, 1.05f);
EXPECT_EQ(featureBorder, 8);
EXPECT_EQ(WEBARKIT_HEADER_VERSION_STRING, "1.0.0");
EXPECT_EQ(WEBARKIT_HEADER_VERSION_STRING, "0.8.0");
}

TEST(WebARKitConfigTest, TestWinSize) {
Expand Down Expand Up @@ -136,7 +136,7 @@ TEST(WebARKitTest, CheckWebARKitVersion) {
// Init the manager with the Akaze tracker
manager.initialiseBase(webarkit::TRACKER_TYPE::AKAZE_TRACKER, 640, 480);
// Check if the WebARKit version is correct
EXPECT_STREQ(manager.getWebARKitVersion().c_str(), "1.0.0");
EXPECT_STREQ(manager.getWebARKitVersion().c_str(), "0.8.0");
}

// Check cameraProjectionMatrix from manager
Expand Down
Loading