MerlinAI-Interpreters

Experiments, interpreter implementations, demos, data ingress tangents and lots of notes for birdsong ID

Web Interpreter Setup:

node:

cd interpreter

# node:
npm install

# venv:
python3 -m venv merlinai_venv
source merlinai_venv/bin/activate
pip3 install -r requirements.txt

# follow the setup friendly prompts:
npm run-script setup-app  # interactive setup, build, config.cfg 

# serve: 
npm run-script serve-app  # serve with default Flask WSGI

Interpreter Flask Structure:

├── app
  ├── main
    ├── annotator
      └── /annotator/ blueprint routes
    ├── auth
      └── token authentication methods, a wip
    ├── classify
      └── config
        └── TensorFlow configuration, POST upload methods, classify blueprint globals
      └── models
        └── classifier class, methods for both Select Ops & standard Ops on server
      └── routes
        └── /classify/ blueprint routes
      └── trashd
        └── garbage collection daemon
    ├── config
      └── use the setup script to populate a new config.cfg file
    ├── eventdb
      └── wip blueprint for ID event database, see notes on this
    ├── tfmodels
      └── Tensorflow model class, routes & whatnot
    ├── tools
      └── utilities for date/time, expression matching, the like
    └── userdb
      └── wip database for user token authentication
├── scripts
  └── scripts run by package.json
├── demos
  └── client side pages, demos built w/ webpack
├── src
  └── source directory for client side demos
├── db
  └── mongodb directory and logs are built here
...


<h4 id=”additional-scripts> </h4>

additional scripts:

build some production binaries:

npm run-script dist-spec-web
npm run-script dist-anno-otf
npm run-script dist-anno-remote
npm run-script dist-anno-photo
npm run-script dist-webgl-web
npm run-script dist-anno-photo

build specific dev bundles:

npm run-script build-spec-web
npm run-script build-anno-otf
npm run-script build-anno-remote
npm run-script build-anno-photo
npm run-script build-webgl-web
npm run-script build-anno-photo

serve more things:

npm run-script setup-app   # interactive Flask setup
npm run-script serve-app   # serve w/ flask
npm run-script serve-node  # serve w/ local http-server + openssl

–watch:

# Webpack with --watch running:
npm run-script watch-spec-web

clean things:

npm run-script clean-bundles
npm run-script clean-renders
npm run-script clean-all

check on waitress config:

# @Procfile:
# Production WSGI sever is waitress;
# check local config with:
waitress-serve --call app:create_app

local ssl:

# Generates local ssl certs for testing w/ node http-server:
npm run-script sslgen

# you can also provide a $DOMAIN argument like so:
npm run-script sslgen hiyori
# ...returns key `hiyori_key.pem` & cert `hiyori.pem`

# ...or:
sudo chmod +x scripts/sslgen.sh && ./scripts/sslgen.sh
# osx is a bit more finicky

tone generator:

### available from here:
cp etc/tone.py .

### generate some .wav files for testing fft things:
python3 tone.py

### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440

### ...or just duration:
python3 tone.py 2

# print reference json strucures:
npx ts-node etc/json_refs.ts

Leaflet Annotator:

This is unlikely to be needed, as the BitBucket annotator tools have moved over to Vue JS and are no longer relevant to this repo To update the leaflet.annotation.js bundle from BitBucket:

To fetch the latest typed leaflet audio annotator source from BitBucket:

curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output ./LeafletSync && chmod +x ./LeafletSync && ./LeafletSync -e UpdateSourceLeafletEnv
# cleanup:
rm LeafletSync

Setup:

# node:
npm install

# If you're building on a Mac,
# you'll need to manually install
# GNU sed and md5sum utilities.
# you could `brew` install them like this:
brew install gsed md5sha1sum

Test with Flask:

# venv:
python3 -m venv merlinai_venv
source merlinai_venv/bin/activate
pip3 install -r requirements.txt

# interactive config.cfg setup:
npm run-script setup-app  

# serve with dev WSGI:
npm run-script serve-app  

# serve with waitress WSGI:
python3 application.py

Test without Flask:

# generates high quality spectrograms on the fly in browser:
google-chrome ./demos/otf_index.html --allow-insecure-localhost --auto-open-devtools-for-tabs
open ./demos/otf_index.html  # use mimetype on Mac

# displays pre-generated spectrograms from ML:
google-chrome ./demos/remote_index.html --allow-insecure-localhost --auto-open-devtools-for-tabs
# open ./demos/remote_index.html # use mimetype on Mac

Verify versions & checksums:

Each leaflet.annotation.js production bundle is accompanied by a leaflet.annotation.js.LICENSE.txt file.

Both these files are prepended with identical dates; one can evaluate when a bundle was built like this:

# bundle:
head -c 42 ./demos/leaflet.annotation.js && echo " */"
#> /* Packed: Tue 19 Jan 2021 09:26:24 PM EST */

# license:
head -c 42 ./demos/leaflet.annotation.js.LICENSE.txt && echo " */"
#> /* Packed: Tue 19 Jan 2021 09:26:24 PM EST */

Furthermore, leaflet.annotation.js.LICENSE.txt is stamped with the md5 checksum of its leaflet.annotation.js twin; one might verify a bundles’s checksum like this:

md5sum ./demos/leaflet.annotation.js
#> 8fa701125c2b77cfb4c97b4dbaaae694

head -c 120 ./demos/leaflet.annotation.js.LICENSE.txt | tail -c 42 && echo
#> '8fa701125c2b77cfb4c97b4dbaaae694

Directory Tree:

├── app
  ├── main
    ├── client
      └── blueprint routes for packed web demos;
          `tfjs` blueprint class & subroutes are
          registered here too
    ├── config
      └── setup-app script populates a new config.cfg file for Flask.
    ├── tfmodels
      └── `tfjs` model class & blueprint routes are in here
    ├── tools
      └── misc. functions for dates, expression matching, etc
├── demos
  ├── audio_example_task
    └── example annotation task.
  └── client side demos implementing leaflet.annotation.js.
├── scripts
  └── scripts run by package.json.
├── src
  └── annotator class is declared in `annotation_tool.tsx`;
      named exports for client side spectrogram, tfjs, audio
      methods & bundle's logger are declared in `index.ts`
└── webpack
  └── `*_dev.ts` modules exports bundles with dev tools, source map, etc
      `*_dist.ts` builds production bundles- preforms Terser tree shaking, css minification, etc
demos
On-The-Fly Spectrogram Annotator demo 		demos
Remote Spectrogram Annotator demo

LeafletSync:

A ridiculous Chindōgu utility prompt & CLI for fetching private releases & files from GitHub & BitBucket

curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output LeafletSync && chmod +x LeafletSync && ./LeafletSync

./LeafletSync

LeafletSync: Do you want to load values from a file?

If so, enter one now...:[Nope!]:  

 _                 __ _      _     _____                    
| |               / _| |    | |   /  ___|                   
| |     ___  __ _| |_| | ___| |_  \ `--. _   _ _ __   ___  
| |    / _ \/ _` |  _| |/ _ \ __|  `--. \ | | | '_ \ / __|
| |___|  __/ (_| | | | |  __/ |_  /\__/ / |_| | | | | (__   
\_____/\___|\__,_|_| |_|\___|\__| \____/ \__, |_| |_|\___|  
 \                      _____________________/ |             
  \ Fetch from Github: /        α wιρ Σ ♥ |_@__Jess          
  /───────────────────/
  \ Your API Token    | -t |  --token | Required | = <personal-api-token>
   | Your Handle      | -u |  --user  | Required | = <You>
   | Source Repo      | -r |  --repo  \ Required  \ = <RepoName>
   | Repository Owner | -a |  --author \ Required  \ = <TheOwner>
   | Release Version  | -v |  --version | Optional | = Fetch Everything
  / Output Directory  | -o |  --out    / Optional  / = ./dist/
 /─────────────────────────/
 \ Fetch from BitBucket:  /                                     
  \──────────────────────/                                   
   \  Your Handle       / -bu  /  --b-user  / ~Required | = <You>
    \ Your Passhrase   / -bp  / --b-pass   / ~Required / = <personal-api-token>
     \ Source Branch  / -bb  / --b-branch / ~Optional / = master
      \ Source File  / -bf  / --b-file   / ~Optional / = <Fetch Everything>
       \────────────/

Your Handle [<You>]:

Source Repo [<RepoName>]:

Repo Owner [<TheOwner>]:

Host: GitHub | BitBucket [GitHub]:

Your Token [<personal-api-token>]:

Release to fetch: [<v0.0.1>]:

Output to fetch (e.g. /dist/*): [<dist/>]:

...

Swift Native:

# Hack on Swift stuff:
npm run-script develop-swift-demos

Other Linker Libraries:

Project:

$(inherited)
-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"

Target:

-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"


# niftily switch between xcode versions:
sudo xcode-select --switch ~/Downloads/Xcode-beta.app

Interpreter Operations:

Hack on fft functions: *./etc/tone.py:*

### copy from here:
cp etc/tone.py .

### generate some .wav files for testing fft things:
python3 tone.py

### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440

### ...or just duration:
python3 tone.py 2

some fft-related links

React Native:

Install & link Cocoa depends to get cracking on the ios demos:

# link ios depends:
cd ios && pod install && cd ..

Notes:

Rolling notes and whatnot are appended to bottom of readme

Macaulay Annotation:

Field Annotation:

Script stuff:

Web stuff:

Other stuff:

demos, annotators *Π competitiveness*

a single annotation as first class entry in database:


id category_id supercategory media_source attribution bbox user_id
unique annotation identifier species, etc family, genus, etc url to media being annotated; for browser spectrogram demo, this is the audio field; otherwise linked via image_id needn’t be Macaulay specific attribution to media source; author; link to media’s license bounding array of annotation box registered individual who made the annotation


todo:

^ Rolling notes and whatnot are appended above ^