Experiments, interpreter implementations, demos, data ingress tangents and lots of notes for birdsong ID
cd interpreter
# node:
npm install
# venv:
python3 -m venv merlinai_venv
source merlinai_venv/bin/activate
pip3 install -r requirements.txt
# build specific things:
npm run-script build-spec-web
npm run-script build-audio-web
npm run-script build-webgl-web
npm run-script build-anno-web
# serve:
npm run-script setup-app # interactive config.cfg setup
npm run-script serve-app # serve with default Flask WSGI
├── app
├── main
├── annotator
└── /annotator/ blueprint routes
├── auth
└── token authentication methods, a wip
├── classify
└── config
└── TensorFlow configuration, POST upload methods, classify blueprint globals
└── models
└── classifier class, methods for both Select Ops & standard Ops on server
└── routes
└── /classify/ blueprint routes
└── trashd
└── garbage collection daemon
├── config
└── use the setup script to populate a new config.cfg file
├── eventdb
└── wip blueprint for ID event database, see notes on this
├── tfmodels
└── Tensorflow model class, routes & whatnot
├── tools
└── utilities for date/time, expression matching, the like
└── userdb
└── wip database for user token authentication
├── scripts
└── scripts run by package.json
├── demos
└── client side pages, demos built w/ webpack
├── src
└── source directory for client side demos
├── db
└── mongodb directory and logs are built here
...
<h4 id=”additional-scripts> </h4>
build some production binaries:
npm run-script dist-spec-web
npm run-script dist-audio-web
npm run-script dist-webgl-web
npm run-script dist-anno-web
serve more things:
npm run-script setup-app # interactive Flask setup
npm run-script serve-app # serve w/ flask
npm run-script serve-node # serve w/ local http-server + openssl
–watch:
# Webpack with --watch running:
npm run-script watch-spec-web
clean things:
npm run-script clean-bundles
npm run-script clean-renders
npm run-script clean-all
check on waitress config:
# @Procfile:
# Production WSGI sever is waitress;
# check local config with:
waitress-serve --call app:create_app
local ssl:
# Generates local ssl certs for testing w/ node http-server:
npm run-script sslgen
# you can also provide a $DOMAIN argument like so:
npm run-script sslgen hiyori
# ...returns key `hiyori_key.pem` & cert `hiyori.pem`
# ...or:
sudo chmod +x scripts/sslgen.sh && ./scripts/sslgen.sh
# osx is a bit more finicky
tone generator:
### available from here:
cp etc/tone.py .
### generate some .wav files for testing fft things:
python3 tone.py
### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440
### ...or just duration:
python3 tone.py 2
# print reference json strucures:
npx ts-node etc/json_refs.ts
To update the leaflet.annotation.js
bundle from BitBucket:
# update:
cd dist && curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output ./LeafletSync && chmod +x ./LeafletSync && ./LeafletSync -e UpdateBundleLeafletEnv
# cleanup:
rm LeafletSync && cd ..
To fetch the latest typed leaflet audio annotator source from BitBucket:
curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output ./LeafletSync && chmod +x ./LeafletSync && ./LeafletSync -e UpdateSourceLeafletEnv
# cleanup:
rm LeafletSync
# node:
npm install
# If you're building on a Mac,
# you'll need to manually install
# GNU sed and md5sum utilities.
# you could `brew` install them like this:
brew install gsed md5sha1sum
# venv:
python3 -m venv merlinai_venv
source merlinai_venv/bin/activate
pip3 install -r requirements.txt
# interactive config.cfg setup:
npm run-script setup-app
# serve with dev WSGI:
npm run-script serve-app
# serve with waitress WSGI:
python3 application.py
# generates high quality spectrograms on the fly in browser:
google-chrome ./demos/otf_index.html --allow-insecure-localhost --auto-open-devtools-for-tabs
open ./demos/otf_index.html # use mimetype on Mac
# displays pre-generated spectrograms from ML:
google-chrome ./demos/remote_index.html --allow-insecure-localhost --auto-open-devtools-for-tabs
# open ./demos/remote_index.html # use mimetype on Mac
Each leaflet.annotation.js
production bundle is accompanied by a leaflet.annotation.js.LICENSE.txt
file.
Both these files are prepended with identical dates; one can evaluate when a bundle was built like this:
# bundle:
head -c 42 ./demos/leaflet.annotation.js && echo " */"
#> /* Packed: Tue 19 Jan 2021 09:26:24 PM EST */
# license:
head -c 42 ./demos/leaflet.annotation.js.LICENSE.txt && echo " */"
#> /* Packed: Tue 19 Jan 2021 09:26:24 PM EST */
Furthermore, leaflet.annotation.js.LICENSE.txt
is stamped with the md5 checksum of its leaflet.annotation.js
twin; one might verify a bundles’s checksum like this:
md5sum ./demos/leaflet.annotation.js
#> 8fa701125c2b77cfb4c97b4dbaaae694
head -c 120 ./demos/leaflet.annotation.js.LICENSE.txt | tail -c 42 && echo
#> '8fa701125c2b77cfb4c97b4dbaaae694
├── app
├── main
├── client
└── blueprint routes for packed web demos;
`tfjs` blueprint class & subroutes are
registered here too
├── config
└── setup-app script populates a new config.cfg file for Flask.
├── tfmodels
└── `tfjs` model class & blueprint routes are in here
├── tools
└── misc. functions for dates, expression matching, etc
├── demos
├── audio_example_task
└── example annotation task.
└── client side demos implementing leaflet.annotation.js.
├── scripts
└── scripts run by package.json.
├── src
└── annotator class is declared in `annotation_tool.tsx`;
named exports for client side spectrogram, tfjs, audio
methods & bundle's logger are declared in `index.ts`
└── webpack
└── `*_dev.ts` modules exports bundles with dev tools, source map, etc
`*_dist.ts` builds production bundles- preforms Terser tree shaking, css minification, etc
![]() On-The-Fly Spectrogram Annotator demo |
![]() Remote Spectrogram Annotator demo |
---|
A ridiculous Chindōgu utility prompt & CLI for fetching private releases & files from GitHub & BitBucket
curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output LeafletSync && chmod +x LeafletSync && ./LeafletSync
FetchReleasegSampleEnv_GitHub
, FetchFilegSampleEnv_BitBucket
, FetchEverythingSampleEnv_BitBucket
, FetchEverythingSampleEnv_GitHub
; pass as an argument with the ` -e ` flag, (./LeafletSync -e YourEnvFile
) or provide one on launch../LeafletSync
LeafletSync: Do you want to load values from a file?
If so, enter one now...:[Nope!]:
_ __ _ _ _____
| | / _| | | | / ___|
| | ___ __ _| |_| | ___| |_ \ `--. _ _ _ __ ___
| | / _ \/ _` | _| |/ _ \ __| `--. \ | | | '_ \ / __|
| |___| __/ (_| | | | | __/ |_ /\__/ / |_| | | | | (__
\_____/\___|\__,_|_| |_|\___|\__| \____/ \__, |_| |_|\___|
\ _____________________/ |
\ Fetch from Github: / α wιρ Σ ♥ |_@__Jess
/───────────────────/
\ Your API Token | -t | --token | Required | = <personal-api-token>
| Your Handle | -u | --user | Required | = <You>
| Source Repo | -r | --repo \ Required \ = <RepoName>
| Repository Owner | -a | --author \ Required \ = <TheOwner>
| Release Version | -v | --version | Optional | = Fetch Everything
/ Output Directory | -o | --out / Optional / = ./dist/
/─────────────────────────/
\ Fetch from BitBucket: /
\──────────────────────/
\ Your Handle / -bu / --b-user / ~Required | = <You>
\ Your Passhrase / -bp / --b-pass / ~Required / = <personal-api-token>
\ Source Branch / -bb / --b-branch / ~Optional / = master
\ Source File / -bf / --b-file / ~Optional / = <Fetch Everything>
\────────────/
Your Handle [<You>]:
Source Repo [<RepoName>]:
Repo Owner [<TheOwner>]:
Host: GitHub | BitBucket [GitHub]:
Your Token [<personal-api-token>]:
Release to fetch: [<v0.0.1>]:
Output to fetch (e.g. /dist/*): [<dist/>]:
...
# Hack on Swift stuff:
npm run-script develop-swift-demos
info.plist
has permissions for microphone access./swift/swift-pkgs-tmpui/swift-pkgs-tmpui/swift_pkgs_tmpuiApp.swift
Project:
$(inherited)
-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"
Target:
-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"
# niftily switch between xcode versions:
sudo xcode-select --switch ~/Downloads/Xcode-beta.app
Hack on fft functions: *./etc/tone.py:*
### copy from here:
cp etc/tone.py .
### generate some .wav files for testing fft things:
python3 tone.py
### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440
### ...or just duration:
python3 tone.py 2
some fft-related links
# link ios depends:
cd ios && pod install && cd ..
./index.js
, fiddle @ ./native/
.Rolling notes and whatnot are appended to bottom of readme
tmpui-testing
dyno used sporadically for debugging, keeping it @ maintenance mode atmheroku buildpacks:add --index 1 heroku-community/apt -a tmpui
for librosa vorbis dependMacaulay Annotation:
Field Annotation:
Script stuff:
Web stuff:
Other stuff:
demos, annotators *Π competitiveness*
a single annotation as first class entry in database:
id | category_id | supercategory | media_source | attribution | bbox | user_id |
---|---|---|---|---|---|---|
unique annotation identifier | species, etc | family, genus, etc | url to media being annotated; for browser spectrogram demo, this is the audio field; otherwise linked via image_id needn’t be Macaulay specific |
attribution to media source; author; link to media’s license | bounding array of annotation box | registered individual who made the annotation |
database of annotations may hold many entries describing the same media, by different users
unregistered users may annotate and play with tools as much as they want, cannot save / contribute their annotations
untrusted (new) registered users may save / contribute their annotations with a confidence level of 1
trusted registered users with more than 5 or more trust points may save / contribute their annotations with a confidence level of 1 or 2 (e.g. duplicate their annotation) (or 0 / just don’t save that annotation)
a user may become trusted if 3 of their annotations contribute to completed annotations
to complete an annotation, entries of X media must be replicated Y times- e.g. identical catagory
, supercatagory
, bbox
centroid is within T threshold of each other (bbox
values can be averaged to single annotation)
users cannot modify existing annotations, that’s cheating
todo:
^ Rolling notes and whatnot are appended above ^