v3: introducing Image Tracking and new AR.js structure

[nicolocarpignoli]

⚠️ All PRs have to be done versus 'dev' branch, so be aware of that, or we'll close your issue ⚠️

🖼 This will make AR.js capable of detect and track 2D images.

https://twitter.com/nicolocarp/status/1204841439160684549

Could be raise some bugs and incompatibilties between the code.

Can it be referenced to an Issue? If so what is the issue # ?
From jsartoolkit5 repo: PR kalwalt/jsartoolkit5#1

From AR.js issues:

• NFT (Natural Feature Tracking) on AR.js jeromeetienne/AR.js#544
• How to scan an image without the black border for marker?  jeromeetienne/AR.js#149
• Markerless AR could be Possible with AR.js jeromeetienne/AR.js#503
• Can i create ar without marker image?I want to tracking the corner to show my mesh,is it possible? jeromeetienne/AR.js#420
• Question about natural feature tracking in ar.js jeromeetienne/AR.js#227
• Is Markerless AR possible with AR.js? jeromeetienne/AR.js#190
• Don't Aframe-AR support NFT? jeromeetienne/AR.js#185

also, a repo cleaning will help for: jeromeetienne/AR.js#433
and also: jeromeetienne/AR.js#722

How can we test it?
Just test the examples on three.js/aframe examples folders.

Summary

see the other PR:
jeromeetienne/AR.js#722
jeromeetienne/AR.js#740

Does this PR introduce a breaking change?

Old repository will continue to be up, with markers + GPS support.

A new repository will be created on AR.js new organization, that will feature two libraries:

• one with marker+gps (as current)
• one with image tracking + gps

Because of incompatibility between markers and NFT.

Please TEST your PR before proposing it. Specify here what device you have used for tests, version of OS and version of Browser

Other information
I uploaded the new jsartookit5 libs and build the AR.js libs with makefile.

Progress

• Clean repo before starting the porting (removing old aruco and tango code)
• Import new jsartoolkit5 version, coming with NFT
• Adapt the code (threejs side) to support NFT (emulate what's already done for fiducial markers)
• Fix the rendering of content on AR.js side, for NFT (onMarkerFound function)
• Make it work on mobile
• Enhance resolution in both mobile and desktop
• Check if animations work with NFT. If not, or rendering rate is too high, use interpolation
• Fix broken pattern marker (content appears shifted after NFT modifies, or maybe even before during the cleaning of the repo)
• Remove all rendering code on the Worker
• Adapt the code (aframe side) to support NFT (emulate what's already done for fiducial markers)
• Create at least one example for threejs + NFT
• Create at least one example for aframe + NFT
• Change documentation (principal Readme, also with NFT Marker generator + Readmes inside threejs + aframe folders + new structure)

[kalwalt]

Testing the Three.js examples:

• webvr-for-ar.html need a fix (for the position of the cube)
• parameters-tunning.html the cube is heavily distorted running in a Mobile device, in desktop is shifted. (this example worked before?)
• markerooot.html example needs a headtracking.mp4 video that doesn't exist anymore in the repo.
• arjs-session.html example cube is in position but grid is not onto the plane of the marker. This needs to be checked.
• nft.html example model is better displayed in portrait mode, in landscape mode is not well centered,

Aframe examples:
basic, dev, minimal and mobile-performance use deprecated camera and so the model appear shifted.
nft example works in similar way as for three.js
nft-video example only show a black plane, i receive this error while loading all the code:

GET https://cors-anywhere.herokuapp.com/https://www.w3schools.com/html/mov_bbb.mp4 429 (Too Many Requests)
(index):1 Uncaught (in promise) 
Event {isTrusted: true, type: "error", target: video#vid, currentTarget: null, eventPhase: 0, …}

and then when detecting the NFT marker:

(index):14 Uncaught (in promise) DOMException: The element has no supported sources.

the black screen could be also caused because my device not support high float ( need precision: mediump in the renderer).

I didn't tested yet the loacation based part.

[kalwalt]

In threex-hittesting-plane.js line 17

AR.js/three.js/src/threex/threex-hittesting-plane.js

Line 17 in 0503261

var geometry = new THREE.PlaneGeometry(20,20,19,19).rotateX(-Math.PI/2)

it should be fixed in this way:

+ var geometry = new THREE.PlaneGeometry(20,20,19,19)
- var geometry = new THREE.PlaneGeometry(20,20,19,19).rotateX(-Math.PI/2)

[kalwalt]

For the aframe examples that use deprecated camera types may we use the <a-entity camera></a-entity> instead?

[nicolocarpignoli]

yes, already done. examples are fixed, should need other tests though.

[nicolocarpignoli]

I'm thinking on how to create the new Readme. I'd like to make something simple, as the principal readme, and then create 'sub readme' for each feature. So the principal readme will be like:

• introduction + logo
• disclaimer: link to old repository + thanks to jerome etienne and similar stuff
• what provides AR.js: image tracking + location based + marker tracking: some GIFS + links to sub-readmes
• info about integration with JS frameworks (link to issues, discussions)
• link to other resources (changelog, etc.)
• device support, with specification for every feture.

marker readme:

• snippet
• link to examples
• doc
• link to marker generator
• link to articles
• technology behind

image tracking readme:

• snippet
• link to examples
• link to articles
• link to image descriptors generator
• technology behind

location based readme:

• snippet
• examples
• doc
• articles
• technology behind