A simple key value store with a concept of objects and an easy interface for sripting. Originally intended for desktop use, I currently use it for my smarthome. https://slatecave.net/creations/ddb/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Slatian 470e8ff17a Do some manual array splicing for backwards-compatibility (old debian on my raspi) 4 months ago
src Do some manual array splicing for backwards-compatibility (old debian on my raspi) 4 months ago
tools Project ddb created! 1 year ago
.gitignore Project ddb created! 1 year ago
README.md Documented the --try-ignore-self option 5 months ago
build Project ddb created! 1 year ago
install.sh Removed legacy scripts 5 months ago
meson.build Project ddb created! 1 year ago
rename.sh Project ddb created! 1 year ago
run Project ddb created! 1 year ago
update_src_build_files Project ddb created! 1 year ago

README.md

ddb (desktop database)

DDB is a simple in memory key value store with the goal to serve as IPC infrastructure between simple and less simple scripts.

Usage can include piping system usage information into multiple receiver scripts, storing the last wheater report for offline usage or exposing an interface for a background service. A bit like dbus but a lot simpler.

Using it

Usage: ddb <command>
Avalable commands are:
	help - show this text
	server - starts a ddb server
	cat - starts a ddb client in cat mode
	read_value <o> <k> <<options>> - reads a value from ddb to stdout
	write_value <o> <k> <v> <<options>> - write a value to ddb from argument
	value <o> <k> <<options>> - blank value operation, use with options
		--subscribe - enters a loop that prints updates (after the initial query)
		--no-output-on-unset - does not print an update if value gets unset
		--initial-read - does an initial read on the value (default for read_value)
		--no-initial-read - disables the initial read on the value
		--pipe-out - equivatent to --initial-read and --subscribe
		--pipe-in - starts a read loop that writes values to stdin
		--no-unset - ignores empty lines on stdin
		--try-ignore-self - Try to ignore events triggered by own events
			Useful when used with --pipe-in and --pipe-outs
	wrap <<command>> - wraps a command in a ddb connection
DDB takes arguments via envoirenment variables:
	DDB_SOCKET_PATH: The path to the ddb unix socket
	DDB_PORT: The port ddb will listen on
	DDB_GREETING: The greeting ddb will accept

Note: If DDB_SOCKET_PATH is present it will, for clients
      be the preferred and only way to connect to ddb.

Older versions of ddb had no builtin client and always start the server and ignore the commandline. make sure to add the server command, the behaviour that the server is started by default will be replaced with the bahaviour of ddb --help some time in the future.

Very old versions of ddb accept the tcp port and greeting as command-line parameters, this is no longer supported, make sure you update your launcher scripts when updating.

Starting The Server

To start the ddb server use ddb server and set the envoirenment variables DDB_PORT or DDB_SOCKET_PATH, and DDB_GREETING (basically an access token/password).

The server will listen on the specified tcp-port and/or unix-socket for connections. The tcp-port will only accept connections from the local machine (127.0.0.1 to be specific).

In versions prior to 2022-03-23 you always had to manually clean up the unix socket after stopping the ddb server, it now does this by itself if it manages to successfully set up the socket (if not, it's very likely that the socket is already there and has a good reason for existing) and gets stopped with a SIGINT (Ctrl+c) or SIGTERM.

Connecting to the Server

To connect to a ddb server one simply opens a socket connection to it, transmits the greeting and then starts writing commands, this can be done in almost any language with raw sockets or tools like netcat, however to make life a bit easier ddb comes with its own client functionality wich takes care of the greeting and connecting part.

All builtin methods of connection have in common that they read the connection information from the DDB_PORT, DDB_SOCKET_PATH and DDB_GREETING envoirenment variables (same as the server). If DDB_SOCKET_PATH is set the client will only try to connect via unix socket. In the future there will probably be a DDB_CONNECTION_TYPE variable.

ddb cat

The simplest to use and the most useful for debugging.

ddb cat behaves a bit like netcat in that it almost directly connects it standard-io with the underlying network socket.

Unlike netcat it knows to close the connection after sending a quit message wich helps a bit when some tools don't know when to close a connection.

ddb wrap

The one most likley used for any logic that reacts to ddb events.

ddb wrap is the inverse of ddb cat in that it spwns a new process with the given commands and connectes the standard-io of that process to the ddb socket. What you can manually type into ddb cat will most likely work in an automated way via ddb wrap. Stderr will not be affected by ddb wrap, it can be used for logging or otherwise communicating with the outside world.

NOTE: ddb wrap will wait with spawning the process until it receives a Hello! from the server.

Example: ddb wrap lua some_script_that_speaks_ddb.lua

NOTE: Lua 5.1 for some reason does not work with ddb wrap Lua 5.2 and above does.

ddb read_value

Useful in bash scripts that just have to read some values

ddb read_value takes an object name and a key as arguments, reads the correspoding value from the ddb server, writes it to stdout and exits. If an error occours it writes nothing to stdout, if the value isnot set it writes an empty line.

Example: ddb read_value testobject fookey

Return codes

  • 1: One or multiple arguments or envoirenment variables are missing or have an invalid value.
  • 2: Can not connect to the server at all or Server can't open sokcet.
  • 3: Someone sent invalid data.
  • 4: An ERROR was returned
  • 5: An Error occoured while calling or trying to call an external tool

Protocol

The protocol ddb uses is line based meaning that every command is terminated by a newline caracter.

Greeting

To initialize the connection the client has to transmit the specified greeting followed by a newline, if the greeting is correct the server will respond with the line Hello! after wich it will accept commands.

Objects

Before talking about the protocol the functionality of the server should be explained. DDB assings all key-value pairs to objects so that there are no global variables. These objects can be used to store values. There is alos the possibility to transmit signals for objects, the intended use for them are commands and indications that the data written to the object is now complete, signals can't transmit data outside their name. Objects can also be subscribed to by any client to get notified of property changes and signals. Also notworth is that subscriptions and signals are independent of an objects existance, one could for example request an update to an object that doesn't exist and then observe the object being created as the data comes in. Objects always have a type property, if the type is set and object exists if not it doesn't.

Unlike bus ddb has no way of enumarating objects, this is to prevent name based autodetection, make your tools configurable, this is intended for tinkerers.

Commands

Commands are single lines in the following format:

<type> <object> <key> <value>

The command documentation below uses the above labels.

The response to a command is either OK or ERROR, note that notifications may appear between the command being sent and the response, simple clients can ignore the responses.

Set (Type:>)

The set command sets the property of an objects and notifys all subscribed clients by sending them the set command. It will fail if the object it references does not exist.

NOTE: There is no way to escape a string built into the protocol, if you want to store a list of objects use a space " " caracter as a delimiter (you can't use them in the names), for other data consider using a encoing that suits your needs, but try to avoid storing a lot of data, your application isn't the only one on the system.

Unset (Type:u)

The unset command will null a value assinged to the given key. It will fail if an object does not exist.

Request (Type:r)

The request command will result in a valid set command for the requestwed key being sent back. If the object does not exist it will result in an error.

Signal (Type:s)

The signal command uses the key argument as a signal name, all subscribed clients will receive the signal command. The value argument can not be used with signals.

Subscribe (Type:+)

Subscribes the sending client to an object

How to build?

Dependencies

  • A working vala compiler
  • glib-2.0
  • gio-2.0
  • meson

If you wannt to use all the scripts (not just build and run) you also need lua 5.x installed

A guaranteed up to date list of dependencies can be found in the src/meson.build file.

If you added or removed source files use the update_src_build_files script, it will atomatically update the src/meson.build file so you don't have to do anything.

Building and running

To build it, run the build.sh script, which will automatically setup the build folder, run ninja, and put the output in the projects root directory. The produced binary should be executable, now.

To make development easy, the run script calls the build.sh script and then runs whatever is at the output (it will try and fail if there is none).