Stored-State-Distributed-Interpreter

revolutionizing uptime and scalability

</>

ThingsDB is a platform designed to unify data and logic.

We aim to replace traditional SQL databases and message brokers, offering a streamlined platform for scalable, robust, and real-time applications.

Distributed Architecture

ThingsDB is a distributed system that spreads data and computations across multiple nodes. This architecture provides several advantages, including scalability, fault tolerance, and high availability.

Stored State

ThingsDB stores the state of the interpreter and data across all nodes.
This ensures that the system remains consistent even if a node fails.

Intelligent Data Routing

ThingsDB intelligently routes queries to the most appropriate nodes, based on data locality and processing load. This optimizes performance and reduces network traffic.

Event-Driven Programming

ThingsDB supports event-driven programming, allowing applications to subscribe to events and react to changes in data. This makes ThingsDB ideal for real-time applications.

Modular Architecture

ThingsDB is designed with a modular architecture, which allows developers to extend the system with custom plugins and modules.

ThingsDB is a programming language that uniquely combines data storage and retrieval with state persistence. Unlike other interpreted languages where variables are lost upon termination, ThingsDB maintains its state, ensuring continuous data access and manipulation. Its syntax is similar to JavaScript and Python, making it accessible to a wide range of developers.

What is ThingsDB?

ThingsDB is a programming language that uniquely combines data storage and retrieval with state persistence. Unlike other interpreted languages where variables are lost upon termination, ThingsDB maintains its state, ensuring continuous data access and manipulation.

Its syntax is similar to JavaScript and Python, ensuring an easy learning curve.

Why Choose ThingsDB?

  • Unified Solution: ThingsDB integrates relational data storage, a built-in publish/subscribe (pub/sub) system for real-time event handling, and scheduled task automation, eliminating the need for multiple disparate tools.

  • Real-time Capabilities: With "Rooms," ThingsDB offers lightweight, dedicated communication channels for event-driven updates, ensuring instant data updates for connected clients.

  • Scalability & Robustness: Designed for high availability, ThingsDB runs across multiple nodes, distributing tasks like garbage collection and backups without blocking queries, ensuring uninterrupted service and data consistency.

  • Developer Friendly: The platform's intuitive language and extensive client connectors (Python, C#, Go, PHP, JavaScript/Node.js) facilitate easy adoption and development.

Key Features & Capabilities

  • Procedures: Define and execute custom server-side logic directly within the database.

  • Types: Enforce strict data structures for "things" (objects with properties), ensuring data integrity and optimizing memory usage by storing property names centrally within type definitions.

  • Events & Rooms: Implement real-time data updates and communication, enabling interactive applications and dashboards.

  • Tasks: Schedule code execution for future or recurring operations, providing a robust mechanism for automation.

  • Modules: Extend ThingsDB's core functionality to interact with external services, such as sending HTTP(S) requests for notifications (e.g., NTFY) or connecting to other databases.

  • Relations (Two-Way Links): Simplify complex data relationships (one-on-one, one-to-many, many-to-many) between types, automatically maintaining consistency across linked data.

  • Sets: Manage unordered collections of unique "things," supporting operations like union, intersection, and difference.

  • Control Responses with Wrap-Only Types: Customize the structure of data returned in query responses without altering the underlying data model, enhancing clarity and reducing bandwidth.

  • Flags, Enumerators, and Regex: Leverage bitwise flags for efficient storage of boolean information, define finite sets of valid values with enumerators, and use regular expressions for powerful string validation and manipulation.

  • Backup & Restore: Safeguard your data with non-intrusive, scheduled backups and flexible restoration options for disaster recovery, including support for Google Cloud Storage.

  • HTTP API: Interact with ThingsDB using a familiar HTTP interface, enabling integration with webhooks and other applications when direct client connections are not feasible.

Example code

A compelling demonstration of ThingsDB's power as a platform can be seen in the development of a multi-user To-Do application, showcasing its ability to combine data persistence, structured data, automated relationships, real-time updates, and scheduled automation all within a single system.

Persistent & Structured Data with Automated Relationships

ThingsDB allows you to define custom data structures, referred to as "types," which are persistent within a collection. These types can also have built-in "relations" that automatically manage connections between different data entities.

// Create the 'Todo' and 'User' types
new_type("Todo");
new_type("User");

// Define the 'Todo' type
set_type("Todo", {
    body: "str<1:>",           // The to-do description (must be at least 1 character)
    creation: "datetime",      // When the to-do was created
    completion: "datetime?",   // Optional: When the to-do was completed (nillable datetime)
    user: "User?",             // User who created the to-do 
    id: "#",                   // Expose the internal ID as 'id' property in responses
});

// Define the 'User' type
set_type("User", {
    name: "str",
    todos: "{Todo}",           // A set of Todo items belonging to this user
});

// Establish a bi-directional one-to-many relation between Todo and User
// This means:
// - When a Todo is added to a User's 'todos' set, its 'user' property is automatically set to that User.
// - When a Todo's 'user' property is changed, it's automatically removed from the old User's 'todos' set and added to the new one.
mod_type("Todo", "rel", "user", "todos");

In this example:

  • We define Todo and `User types with specific property definitions, enforcing data integrity and automatically assigning default values.

  • The completion property is datetime?, indicating it's an optional datetime value or nil, allowing for flexibility.

  • The id: "#` property definition ensures that the internal unique ID of a "thing" is exposed as id in responses, providing a consistent and user-friendly identifier.

  • The mod_type("Todo", "rel", "user", "todos") statement sets up a powerful one-to-many relationship. This means that when a `Todo` is assigned to a `User`'s `todos` set, ThingsDB automatically manages the inverse relationship, linking the `Todo` back to its `User` without manual intervention, and ensuring data consistency across the two-way link.

Encapsulated Logic with Real-time Updates (Procedures & Events)

ThingsDB allows developers to create "procedures" which are named closures that encapsulate complex logic and can be executed directly. These procedures can interact with data and also emit "events" to "rooms", enabling real-time communication within the application.

// Place to store users
.users = {};

// Room where dashboard(s) can listen to for changes
.dashboard = room();

// Add a procedure to add a to-do for a specific user
new_procedure("add_todo", |name, body| {
    "Adds a to-do to a user's todo list."; // Docstring describing the procedure's purpose

    todo = Todo{body:,}; // Create a new Todo instance
    .users[name].todos.add(todo); // Add the todo to the specified user's todo set
    
    // Emit an event to the dashboard room to signal a new to-do has been added
    // Assumes .dashboard is a room property on the Root type.
    .dashboard.emit("add-todo", todo.user.id()); 
    
    todo.id(); // Return the ID of the newly created todo
});

  • The add_todo procedure takes a user name and body as arguments.

  • wse() (with-side-effects) is used to explicitly inform ThingsDB that this procedure modifies data, which is crucial for maintaining data consistency across nodes.

  • After creating and adding a Todo to a user's set, the procedure uses .dashboard.emit("add-todo", todo.user.id()) to send a real-time event. This allows external clients subscribed to the "dashboard" room to instantly update their views, demonstrating ThingsDB's built-in pub/sub system.

Scheduled Automation (Tasks)

ThingsDB includes a "task" type that enables scheduling code execution at specific dates and times, ensuring guaranteed consistency and user context execution across nodes.

// Add example users Alice & Bob
.users.Alice = User{name: 'Alice'};
.users.Bob = User{name: 'Bob'};

// Schedule a task to send a daily report reminder
task(datetime(), |task, name, body| {
    task.again_in("days", 1); // Reschedule the task to run again in 1 day
    add_todo(name, body);           // Add the "Send report to Bob" to-do
}, ["Alice", "Send report to Bob"]);

  • This code creates a task that executes immediately (datetime()).

  • Inside the task's closure, task.again_in("days", 1) reschedules the task to run every day, automatically creating a persistent, recurring reminder without external cron jobs or schedulers.

  • The add_todo(name, body) call within the task reuses the previously defined procedure, demonstrating reusability and integration.

Controlling Response Format (Wrap-Only Types)

ThingsDB provides "wrap-only types" to define how existing data should be presented in responses, allowing for flexible output formatting without altering the underlying data structure.

// Define a wrap-only type for presenting Todo data
new_type("_Todo", true); // The 'true' flag indicates it's a wrap-only type
set_type("_Todo", {
    id: "#",                        // Include the original ID
    body: "any",                    // Include the body as is
    done: |this| is_datetime(this.completion), // Computed property: true if completion is set
});

.users.Alice.todos.map_wrap('_Todo');  // wrap the to-do's of Alice 
/* Expected output (example):
[
    {
        "body": "Send report to Bob",
        "done": false,
        "id": 12
    }
]
*/

  • The _Todo type is defined as wrap-only (true argument in new_type). It cannot be instantiated directly but acts as a template for presentation.

  • The done property is a "computed property" (|this| is_datetime(this.completion)). Its value is dynamically calculated based on the `completion` property of the original Todo object, transforming data for display (e.g., from a datetime to a boolean done flag).

    These interconnected code examples highlight ThingsDB's power as a unified, stateful, and real-time data management and application platform, enabling developers to build complex, responsive applications with a simplified architecture.

Community & Resources

  • ThingsDB Website: https://thingsdb.io

  • ThingsDB Documentation: https://docs.thingsdb.io

  • ThingsDB Source Code: https://github.com/thingsdb

  • ThingsDB Discussion Forum: https://github.com/orgs/thingsdb/discussions

  • The ThingsDB Book: https://docs.thingsdb.io/v1/getting-started/book/.

    • See page 11 on how to use Docker Compose to easily deploy your first ThingsDB cluster

    • See page 212-214 to learn about our official Client Connectors for Python, C#, Go, PHP, and JavaScript/Node.js

    • See page 10 and 22 to learn how to interact with ThingsDB using the commandline tool things-prompt or through a more visually appealing graphical interface with ThingsGUI