App Engine app for streaming a music collection
 
 
 
 
 
 
Go to file
Daniel Erat 3d9f964205
web: Use textContent instead of innerText.
textContent is standardized and doesn't trigger a reflow on
read (since unlike innerText, it doesn't exclude "hidden"
elements):

https://developer.mozilla.org/en-US/docs/Web/API/Node/textContent
https://kellegous.com/j/2013/02/27/innertext-vs-textcontent/

As far as I can tell, none of my code cares about the
difference in behavior, so use the faster one.

https://stackoverflow.com/q/17199958 (2013) warns that
textContent may be slower when setting, but textContent
now seems to be almost 4 times faster (Chrome 118):

https://www.measurethat.net/Benchmarks/ShowResult/486502

same value, innerText                    1,737,885 ops/sec
same value, textContent                  6,775,336 ops/sec
same value, nodeValue                    4,942,868 ops/sec
same value if changed, innerText         1,107,992 ops/sec
same value if changed, textContent       7,485,101 ops/sec
same value if changed, nodeValue         10,595,588 ops/sec

different value, innerText               1,731,098 ops/sec
different value, textContext             6,572,991 ops/sec
different value, nodeValue               3,845,782 ops/sec
different value if changed, innerText    1,029,156 ops/sec
different value if changed, textContent  7,739,438 ops/sec
different value if changed, nodeValue    10,500,733 ops/sec

I'm not using the nodeValue approach since I suspect it
won't work if I'm trying to update a div that doesn't
already have a text node child.

I'll probably keep the "if changed" checks that I have in
various position-updating code since it apparently doesn't
cost anything and seems like it might still help prevent
paints (I have no idea how it could be faster in the
"different value case, though).
2023-12-03 11:07:45 -04:00
build build: Switch to Artifact Registry and single-region bucket. 2023-12-01 15:32:17 -04:00
cmd/nup cmd/nup/client: Replace CountBools with CheckNumFlags. 2023-12-02 10:30:06 -04:00
example test/e2e: Test the server's /cover endpoint. 2023-11-23 10:14:09 -04:00
server test/e2e: Test the server's /cover endpoint. 2023-11-23 10:14:09 -04:00
test web: Use textContent instead of innerText. 2023-12-03 11:07:45 -04:00
web web: Use textContent instead of innerText. 2023-12-03 11:07:45 -04:00
.gcloudignore web: Add Fontello config. 2022-09-04 17:14:50 -04:00
.gitignore server: Make /cover and /song return 404 for missing files. 2022-04-21 08:03:15 -04:00
.prettierrc web,server: Route song and cover requests through server. 2021-02-23 20:38:36 -04:00
LICENSE Add LICENSE file. 2021-11-19 07:31:11 -04:00
README.md Update README.md to use screenshots from www.erat.org. 2023-04-18 11:00:52 -04:00
app.yaml Update app.yaml to use go121 runtime. 2023-10-24 11:34:46 -04:00
cron.yaml server: Add /stats endpoint. 2022-02-04 12:35:57 -04:00
go.mod Update still more Google packages in go.mod. 2023-11-13 11:55:58 -04:00
go.sum Update still more Google packages in go.mod. 2023-11-13 11:55:58 -04:00
index.yaml index.yaml: Add index on RatingAtLeast1 and LastStartTime. 2023-07-22 07:16:59 -04:00

README.md

nup

Build Status

App Engine app for streaming a music collection.

Overview

light mode screenshot dark mode screenshot

This repository contains a server for serving a personal music collection, along with a web client and a command-line program for managing the data.

The basic idea is that you mirror your music collection (and the corresponding album artwork) to Google Cloud Storage and then run the nup update command against the local copy to save metadata to a Datastore database. User-generated information like ratings, tags, and playback history is also saved in Datastore. The App Engine app performs queries against Datastore and serves songs and album art from Cloud Storage.

An Android client is also available.

This project is probably only of potential interest to people who both buy all of their music and are comfortable setting up a Google Cloud project and compiling and running command-line programs, which seems like a very small set. If it sounds appealing to you and you'd like to see more detailed instructions, though, please let me know!

History

In 2001 or 2002, I wrote dmc, a silly C application that used the FMOD library to play my MP3 collection. It used OpenGL to render a UI and some simple visualizations. I ran it on a small (Mini-ITX? I don't remember) computer plugged into my TV.

Sometime around 2005 or 2006, I decided that I wanted to be able to rate and tag the songs in my music collection and track my playback history so I could listen to stuff that I liked but hadn't heard recently, or play non-distracting instrumental music while reading or programming. I was using MPD to play locally-stored MP3 files at the time, so I wrote some Ruby scripts to search for and enqueue songs and display information about the current song onscreen. I also wrote a Ruby audioscrobbler library for sending playback reports to the service that later became Last.fm.

In 2010, I decided that it was silly to need to have my desktop computer turned on whenever I wanted to listen to music, so I wrote a daemon in Ruby to serve music and album art and support searching/tagging/rating/etc. over HTTP. Song information was stored in a SQLite database. I added a web interface and wrote an Android client that supported offline playback, and ran the server on a little always-on SoC Linux device. This was before the Raspberry Pi was released, and all I remember about the device was that upgrades were terrifying because it didn't put out enough power to be able to reliably boot off its external HDD.

In 2014, I decided that it'd be nice to be less dependent on my home network connection, so I rewrote the server in Go as a Google App Engine app that'd serve music and covers from Google Cloud Storage. That's what this repository contains.

It's 2021 now and I haven't felt the urge to rewrite all this code again.

The name "nup" doesn't mean anything; it just didn't seem to be used by any major projects. (I tried to think of a backronym for it but didn't come up with anything obvious other than the 'p' standing for "player".)

Configuration

At the very least, you'll need to do the following:

  • Create a Google Cloud project.
  • Enable the Cloud Storage and App Engine APIs.
  • Create Cloud Storage buckets for your songs and album art.
  • Use the gsutil tool to sync songs and album art to Cloud Storage.
  • Compile the nup tool using go install ./cmd/nup.
  • Deploy the App Engine app.
  • Write config files for the nup tool and the app.
  • Use nup update to send song metadata to the App Engine app so it can be saved to Datastore.

As mentioned above, please let me know if you're feeling adventurous and would like to see detailed instructions for these steps.

nup tool

Create a JSON file at $HOME/.nup/config.json corresponding to the Config struct in cmd/nup/client/config.go:

{
  "serverUrl": "https://my-project-id.appspot.com",
  "username": "tools",
  "password": "my-tools-password",
  "coverDir": "/home/me/music/.covers",
  "musicDir": "/home/me/music",
  "computeGain": true
}

Server

Create a JSON file corresponding to the Config struct in server/config/config.go:

{
  "users": [
    { "email": "example@gmail.com" },
    { "email": "example.2@gmail.com" },
    { "username": "android", "password": "my-android-password" },
    { "username": "tools", "password": "my-tools-password", "admin": true }
  ],
  "songBucket": "my-songs",
  "coverBucket": "my-covers"
}

Run nup config -set /path/to/server_config.json to save the server configuration to Datastore.

Deploying

To deploy the App Engine app (as defined in app.yaml) and delete old, non-serving versions, run the build/deploy.sh script.

Note that App Engine often continues serving stale versions of static files for 10 minutes or more after deploying. I think that this has been broken for a long time. This Stack Overflow question has more discussion.

After changes to index.yaml, run build/deploy.sh -i to create new Datastore indexes and delete old ones.

You should also run build/deploy.sh cron.yaml once to deploy the daily stats-updating cron job described in cron.yaml.

Development and testing

The example/ directory contains code for starting a local App Engine server with example data for development.

All tests can be executed by running go test ./... from the root of the repository.

  • Unit tests live alongside the code that they exercise.
  • End-to-end tests that exercise the App Engine server and the nup executable are in the test/e2e/ directory.
  • Selenium tests that exercise both the web interface (in Chrome) and the server are in the test/web/ directory. By default, Chrome runs headlessly using Xvfb.