dfx is a Linux-only dynamic dataflow execution platform for building, running, and managing directed graphs of computational nodes that exchange messages asynchronously.

It targets users who care about explicit dataflow, runtime reconfiguration, process-level control, and low-level system integration - not monolithic pipelines or opaque schedulers.

Written in C++23, built on Linux primitives (epoll, Unix domain sockets, eventfd). Windows is not a target and never will be.

API documentation (Doxygen): doc.dfxengine.com

What problem dfx solves

dfx is designed for scenarios where:

computation is naturally expressed as a graph of independent nodes,
nodes communicate via typed, asynchronous message channels,
the graph must be inspectable, controllable, and modifiable at runtime,
execution must scale across threads and CPU cores,
tooling and observability matter as much as raw throughput.

Typical use cases: stream processing, event pipelines, message routing and transformation, simulation, and experimentation with dynamic execution models.

Core concepts

Concept	Description
Node	Stateful computation unit with named input/output ports. Executes one message at a time.
Port	Typed endpoint on a node. Inputs receive messages; outputs emit them.
Channel	Directed connection from one port to another. Defines message flow.
Graph	A collection of nodes and channels, loaded from a .dfx file.
Message	Typed payload flowing through a channel from one node to another.
Plugin	Dynamically loaded shared library that registers new node and transport types.

Execution is fully asynchronous and event-driven. There is no global clock and no assumed execution order. Nodes run on a shared thread pool, with at most one thread active inside any given node at a time. The runtime prevents stack overflows in deep or cyclic graphs by bounding inline execution depth.

Binaries

dfx is composed of four cooperating executables.

dfx - Runtime executor

Loads a .dfx graph, wires nodes and channels, and runs the graph asynchronously. Exposes a Unix domain socket control interface for runtime inspection and modification.

dfx [OPTIONS] [graph-file]
 
  -c, --config <path>       Override the TOML config file (default: config.toml)
  -l, --log-level <level>   Set global log level (trace/debug/info/warning/error/critical/off)
      --no-control          Disable the control socket
      --control-socket <p>  Override the control socket path
      --trace-file <path>   Write PcapNG message trace to a file
      --trace-pipe <path>   Create a named pipe for live PcapNG streaming (one-shot, see note)
      --trace-tcp <addr>    Expose a TCP server for live PcapNG streaming (e.g. localhost:9999)

Graphs can be modified at runtime - nodes added or removed, channels rewired - without stopping execution.

dfxd - Daemon and instance manager

A long-running supervisor that spawns, monitors, and terminates dfx processes. Provides a stable control plane over a Unix domain socket. Does not execute graphs itself.

dfxd [OPTIONS]
 
  -c, --config <path>       Override the TOML config file (default: config.toml)
  -l, --log-level <level>   Set global log level
      --control-socket <p>  Override the control socket path

Each managed instance has its own graph, runtime state, control socket, and lifecycle. dfxd tracks all instances and applies restart policies independently.

dfx-ctl - Control and inspection CLI

The command-line interface for interacting with running dfx instances and the dfxd daemon. Stateless: each invocation connects, sends commands, receives a response, and exits - similar in spirit to docker or kubectl.

dfx-ctl [OPTIONS] SUBCOMMAND
 
  -i, --instance <path>     Path to the target dfx socket (default: /tmp/dfx.sock)
  -f, --format <fmt>        Output format: text or json
      --pretty              Pretty-print JSON output
  -q, --quiet               Suppress non-essential output

Subcommand groups:

Group	Commands
node	list, get, add, remove
channel	list, get, add, remove
graph	export, validate, import
config	get, set
raw	Send raw protocol commands directly to a socket

Examples:

# List all nodes in the running graph
dfx-ctl node list
 
# Add a new Log node while the instance is running
dfx-ctl node add logger --type Log
 
# Connect two nodes
dfx-ctl channel add gen.out logger.in
 
# Export the current runtime graph
dfx-ctl graph export --output graph.dfx
 
# Low-level: send a raw command to the daemon
dfx-ctl raw --target /tmp/dfxd.sock '{"command": "instance/list"}'

dfx-gui - Graphical graph editor

A Qt6-based visual editor for authoring and inspecting dataflow graphs. Focuses on graph design, not execution.

Features:

visual creation and editing of nodes, ports, and channels,
node library panel with categorized, searchable node types,
node inspector with configuration editing (JSON form and raw text),
graphical representation of port connections and message topology,
validation and preparation before saving.

Graphs authored with dfx-gui are saved in the standard .dfx format and can be executed directly by dfx.

dfx-gui [graph-file...]

Architecture overview

┌──────────┐   manages    ┌──────────┐   spawns   ┌───────────────┐
│  dfx-ctl │ ──────────── │  dfxd    │ ─────────── │  dfx (×N)     │
│  dfx-gui │              │  daemon  │             │  graph runner │
└──────────┘              └──────────┘             └───────────────┘
     │                         │                         │
     └─────────────────────────┴─────────────────────────┘
                   Unix domain socket (control plane)

The runtime uses:

a dedicated FD listener thread (epoll) for I/O and timer readiness,
a task-based scheduler for dispatching node work across worker threads,
node-aware serialization to guarantee safe, exclusive execution per node.

Libraries (internal, also usable as Conan components):

Library	Role
dfx-core	Base message, port, and node abstractions
dfx-runtime	Scheduler and execution engine
dfx-graph	Graph loader, controller, and node/transport factories
dfx-server	Unix domain socket server and command routing
dfx-fdwatch	epoll wrapper, signal handling, timer and FD watchers
dfx-plugins	Plugin registry and dynamic library loading
dfx-plugins-interface	C ABI used by plugin shared libraries
dfx-nodes	Built-in node implementations
dfx-client	Client library for connecting to a dfx or dfxd socket
dfx-pcapng	PcapNG capture and streaming for message tracing
dfx-subprocess	Subprocess management for dfxd
dfx-utilities	Logging, configuration, file-system helpers

Graph format

Graphs are stored as JSON with the .dfx extension. The format is straightforward:

{
  "nodes": [
    { "name": "gen", "type": "Generator", "config": { "frequency": 10000 } },
    { "name": "log", "type": "Log",       "config": {} }
  ],
  "channels": [
    { "from": "gen.out", "to": "log.in" }
  ]
}

nodes - list of node instances, each with a unique name, a type registered by the runtime or a plugin, and an optional configuration object.
channels - directed connections in "source-node.port" → "dest-node.port" format.
gui - optional layout metadata consumed by dfx-gui (ignored by dfx).

Graphs can also be generated or inspected with dfx-ctl graph export/validate.

Plugin system

dfx supports dynamically loaded plugins (.so shared libraries). Plugins register new node types and transport types into the runtime at startup.

Plugin configuration lives in config.toml:

[plugins]
search_paths = ["/usr/local/lib/dfx/plugins"]
 
[[plugins.info]]
name = "org.dfx.bridges.lua"
allow_nodes = []
deny_nodes  = []
[plugins.info.config]
search_paths = ["/usr/local/share/dfx/lua"]

Plugins implement the C ABI declared in dfx-plugins-interface and are loaded via dlopen. The plugin registry outlives the scheduler to ensure no plugin code runs after dlclose.

A Wireshark dissector for the dfx control protocol is included in wireshark-dissector/.

Message tracing (PcapNG)

dfx can capture the message flow of a running graph and emit it in PcapNG format, compatible with Wireshark and other packet analysis tools.

# Write a complete trace to a file after execution
dfx --trace-file trace.pcapng my-graph.dfx
 
# Live capture over TCP (recommended for live use)
dfx --trace-tcp localhost:9999 my-graph.dfx &
wireshark -k -i TCP@localhost:9999

Named pipe (--trace-pipe) - one-shot only. A named pipe cannot signal a new connection, so the PcapNG header is only written once when dfx starts. If Wireshark is closed and reconnected, the header will be missing and the capture will fail. Use --trace-tcp for any scenario where you may need to reconnect.

Building

Requirements

Requirement	Minimum version
Linux	(any modern kernel)
GCC	13
Clang	16
CMake	3.21
Conan	2.0
Qt	6.0 (only for dfx-gui)

Dependencies (managed by Conan)

nlohmann/json 3.11.3
valijson 1.0.3
spdlog 1.15.1
CLI11 2.5.0
toml++ 3.4.0
crc32c 1.1.2
Catch2 3.8.1 (tests only)

Steps

# 1. Install dependencies
conan install . --build=missing
 
# 2. Build (runs cmake configure, build, and tests)
conan build .

To build without the Qt GUI:

conan install . --build=missing -o dfx/*:with_qt=False

conan build .

To build without documentation (requires Doxygen by default):

conan install . --build=missing \
    -c "tools.cmake.cmaketoolchain:extra_variables={'DFX_WITH_DOC': 'OFF'}"
conan build .

Configuration

Both dfx and dfxd accept a TOML configuration file (default: config.toml in the working directory, overridable with -c).

[server]
# Path to the Unix domain socket (default: /tmp/dfx.sock for dfx, /tmp/dfxd.sock for dfxd)
socket_path = "/tmp/dfx.sock"
enabled     = true
 
[daemon]
# Path to the dfx binary used by dfxd to spawn instances
dfx.bin_path = "/usr/local/bin/dfx"
 
[plugins]
search_paths = ["/usr/local/lib/dfx/plugins"]

Command-line flags always take precedence over config file values.

Documentation

Full API and architecture documentation is generated with Doxygen and available at:

https://doc.dfxengine.com/

The documentation covers all internal libraries, node and graph abstractions, the runtime control interface, and the IPC/daemon architecture.

To generate documentation locally (requires Doxygen and Graphviz):

conan install . --build=missing
conan build . --target doc
# Output: build/RelWithDebInfo/doc/html/index.html

License

dfx is released under the MIT License. See [LICENSE](LICENSE) for full details.