rodrigo/genieacs
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add ARCHITECTURE.md and AGENTS.md, expand CONTRIBUTING.md

Browse Source 
Add ARCHITECTURE.md with comprehensive codebase map covering service
boundaries, expression system, session engine, sandbox, database layer,
and architectural invariants. Add AGENTS.md as a concise guide for AI
coding agents. Expand CONTRIBUTING.md with detailed code style, naming,
import, testing, and git workflow conventions.
This commit is contained in:
Zaid Humaimidi committed 2026-03-12 14:26:46 -07:00
1 parent cc723c0b50
commit bef840ce0a
3 files changed
+877 -21
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+82
View File
@@ -0,0 +1,82 @@
# AGENTS.md — GenieACS
GenieACS is a TR-069 Auto Configuration Server for remote management of CPE
devices (routers, modems, gateways). TypeScript codebase compiled with esbuild,
backed by MongoDB.
## Architecture Overview
Four services share a single MongoDB instance:
- **CWMP** (port 7547) — TR-069 protocol handler; manages device sessions
- **NBI** (port 7557) — Northbound REST API for external consumers
- **FS** (port 7567) — File server for firmware/config (GridFS-backed)
- **UI** (port 3000) — Web interface (Koa backend + Mithril.js SPA frontend)
Key subsystems: expression engine (`lib/common/expression/`) compiles a
Lisp-like DSL used for queries, config, and authorization; session engine
(`lib/session.ts`) drives CWMP interactions via declarations rather than
imperative RPCs; sandbox (`lib/sandbox.ts`) runs user-defined provision scripts
in `vm.Script` with deterministic replay.
Read `ARCHITECTURE.md` for a full map of the codebase when working on unfamiliar
areas. It covers service boundaries, the expression pipeline, the path system,
the CWMP session state machine, the database layer, and architectural
invariants.
## Project Structure
- `lib/` — Core server-side logic
- `lib/common/` — Shared code (runs in both Node.js and browser)
- `lib/db/` — MongoDB database layer
- `lib/ui/` — UI backend helpers
- `ui/` — Frontend SPA (Mithril.js)
- `bin/` — Service entry points (5 executables)
- `build/` — Build scripts (esbuild pipeline)
- `test/` — Unit tests (node:test)
- `docs/` — User docs (Sphinx/reStructuredText)
- `public/` — Static assets (favicon, logo)
## Build / Lint / Test Commands
```bash
npm run build # Production build (esbuild pipeline -> dist/)
NODE_ENV=development npm run build # Dev build (no minification)
npm run lint # Prettier + ESLint + tsc --noEmit in parallel
npm test # Compile tests with esbuild, run with node --test
```
### Running a Single Test File
```bash
esbuild --log-level=warning --bundle --platform=node --target=node18 \
--packages=external --sourcemap=inline --outdir=test test/path.ts \
&& node --test --enable-source-maps test/path.js \
&& rm test/path.js
```
### Running a Single Test Case
```bash
esbuild --log-level=warning --bundle --platform=node --target=node18 \
--packages=external --sourcemap=inline --outdir=test test/path.ts \
&& node --test --enable-source-maps --test-name-pattern="^parse$" test/path.js \
&& rm test/path.js
```
### Lint Sub-commands
```bash
prettier --prose-wrap always --write .
eslint 'bin/*.ts' 'lib/**/*.ts' 'ui/**/*.ts' 'test/**/*.ts' 'build/**/*.ts'
tsc --noEmit
```
## Before Committing
Read `CONTRIBUTING.md` and ensure your changes comply with it. In particular:
- Run `npm run lint` and `npm test` and fix any failures.
- Follow the code style, naming, import, and comment conventions documented
there.
- Use the Conventional Commits format for commit messages.
ARCHITECTURE.md
+461
View File
@@ -0,0 +1,461 @@
# Architecture
This document describes the high-level architecture of GenieACS. If you want to
familiarize yourself with the codebase, you are in the right place.
## Bird's Eye View
GenieACS is a TR-069 Auto Configuration Server (ACS). It manages CPE devices
(routers, modems, gateways) using the CWMP protocol (TR-069). The system
consists of four network-facing services that share a MongoDB database:
```
+------------+
CPE Devices --->| CWMP (7547)|-+
+------------+ |
+-----------+ | +---------+
CPE Devices --->| FS (7567) |--+---->| MongoDB |
+-----------+ | +---------+
+-----------+ |
OSS / Scripts ->| NBI (7557)|--+
+-----------+ |
+-----------+ |
Administrators->| UI (3000) |--+
+-----------+
```
- **CWMP** -- The core TR-069 protocol server. CPE devices connect here to
report their state and receive configuration instructions via SOAP/XML over
HTTP.
- **NBI** -- Northbound Interface. A REST API for external systems (OSS/BSS,
scripts, automation) to manage devices, tasks, presets, and configuration
programmatically.
- **FS** -- File Server. Serves firmware images and configuration files to CPE
devices during download operations.
- **UI** -- Web interface. A Koa-based backend serving a Mithril.js single-page
application for administrators to browse devices, manage configuration, and
trigger operations.
All four services follow the same process model: a primary process forks
configurable worker processes via Node.js `cluster` (see `cluster.ts`). Workers
connect to MongoDB and start an HTTP(S) server.
## Code Map
```
bin/ Service entry points (5 executables)
lib/ All backend logic
common/ Shared utilities (expression engine, Path, errors)
expression/ Expression parser, evaluator, normalizer, minimizer
cwmp/ CWMP-service-specific DB and caching
db/ Database layer (MongoDB collections, query synthesis)
ui/ UI-service-specific API, DB, and caching
types/ (empty, reserved)
ui/ Frontend SPA (Mithril.js)
components/ Reusable UI components (parameter, tags, ping, etc.)
css/ Stylesheets (vanilla CSS)
icons/ SVG icons (compiled into a sprite)
test/ Unit tests (Node.js native test runner)
build/ Build scripts (esbuild-based)
public/ Static assets (logo, favicon)
```
### Entry Points (`bin/`)
Each file in `bin/` bootstraps one service. They are structurally identical:
initialize logging, read config, fork workers in the primary process, connect to
MongoDB and start the HTTP server in each worker.
- `genieacs-cwmp.ts` -- CWMP service. Unique in that it disables HTTP keep-alive
(`keepAliveTimeout: 0`) and provides custom `onConnection` / `onClientError`
hooks for TR-069 session lifecycle management.
- `genieacs-nbi.ts` -- NBI service. Straightforward REST API server.
- `genieacs-fs.ts` -- File server. The leanest service; does not use the
extensions subsystem.
- `genieacs-ui.ts` -- UI service. Wraps the Koa application as the HTTP
listener.
- `genieacs-ext.ts` -- **Not a service.** This is a child process worker spawned
by the extensions subsystem. It communicates with its parent via IPC,
executing user-defined extension scripts in isolation.
### The Expression System (`lib/common/expression/`)
The `Expression` type (`string | number | boolean | null | any[]`) is the most
important abstraction in the codebase. It is a recursive, Lisp-like S-expression
where scalars are literal values and arrays are operations (the operator at
index 0, operands following). For example:
```
["AND", ["=", ["PARAM", "Device.ModelName"], "BrandX"], [">", ["PARAM", "Events.Inform"], 1000]]
```
Expressions are used pervasively:
- **Query/filter language** -- Database queries for devices, faults, presets,
etc. are all represented as expressions and compiled to MongoDB filters.
- **Configuration values** -- Config entries and preset preconditions are
expressions, enabling dynamic evaluation.
- **Authorization** -- Permission filters and validators are expressions.
- **Pagination cursors** -- Keyset pagination boundaries are expressed as filter
expressions.
The expression pipeline flows through four modules:
1. `parser.ts` -- Parses a SQL-like text syntax into the AST (uses Parsimmon).
Also provides `map()` / `mapAsync()` tree-walking primitives and
`stringify()` for serialization.
2. `normalize.ts` -- Algebraic normalization using exact rational polynomial
arithmetic over bigints. Ensures equivalent expressions have the same
canonical form (e.g., `a + 2 > b` and `a > b - 2` normalize identically).
3. `synth.ts` -- Boolean logic minimization via the Espresso algorithm
(espresso-iisojs). Converts expressions to minimal sum-of-products form with
three-valued logic (true/false/null). Handles domain-specific constraints
like comparison ordering and LIKE pattern relationships.
4. `util.ts` -- Runtime evaluation. Resolves `PARAM` references against concrete
data, evaluates operators, and supports partial evaluation (returns a reduced
expression if some values are unknown).
`bigint.ts` provides a named-function wrapper around native bigint operations
for readability. `pagination.ts` implements cursor-based pagination by
generating filter expressions from sort-key bookmarks.
### The Path System (`lib/common/path.ts`, `lib/common/path-set.ts`)
`Path` represents a TR-069 parameter path (e.g., `Device.WiFi.SSID.1.Name`).
Paths can contain wildcards (`*`) and alias expressions (`[key:value]`) for
query-based addressing.
Key design decisions:
- **Interned/flyweight** -- All `Path` instances are cached. `Path.parse()` is
the only way to create them; the constructor is protected. Two-generation LRU
cache rotated every 120 seconds.
- **Bitmask encoding** -- The `wildcard` and `alias` fields are bitfields for
O(1) segment-type checking. This limits paths to 32 segments.
- **Immutable** -- Segment arrays are `Object.freeze()`-d.
`PathSet` is a multi-indexed collection of paths supporting pattern-matching
queries. It indexes paths by length, by segment-at-position, and by string
representation. The `find()` method uses set intersection to efficiently match
wildcard patterns.
### CWMP Protocol Layer (`lib/cwmp.ts`, `lib/soap.ts`, `lib/xml-parser.ts`)
`xml-parser.ts` is a custom single-pass XML parser (no DOM). It scans
character-by-character with bitwise state flags, building a tree of elements
with namespace support. Does not support CDATA.
`soap.ts` handles both parsing CPE SOAP messages and generating ACS SOAP
responses. It dispatches on the SOAP body's method name to type-specific parsers
for Inform, TransferComplete, GetParameterNamesResponse, etc. Supports CWMP
versions 1.0 through 1.4.
`cwmp.ts` is the HTTP-level CWMP request handler. It manages the session state
machine:
1. **State 0** -- Expects an Inform. Authenticates the device (Basic or Digest
auth, configurable via expression). Acquires a distributed lock. Loads device
data from MongoDB. Sends InformResponse.
2. **State 1** -- Waits for the CPE to send an empty POST (ready for ACS RPCs).
Processes any TransferComplete messages.
3. **State 2** -- The ACS drives RPCs (GetParameterNames, GetParameterValues,
SetParameterValues, AddObject, DeleteObject, Download, Reboot, FactoryReset).
The CPE responds to each.
Session persistence across TCP disconnects: when a socket closes mid-session,
the entire `SessionContext` is serialized to Redis (the MongoDB `cache`
collection) and restored when the CPE reconnects (identified by a session
cookie).
### Session Engine (`lib/session.ts`)
The session engine implements the **declaration-driven data fetching** pattern.
Rather than issuing RPCs imperatively, provisions create `Declaration` objects
stating what paths and attributes they need to read or write. The engine then:
1. Processes all declarations into a `SyncState` -- a structured plan of which
parameters to refresh, which values to set, which instances to create/delete,
etc.
2. Generates the minimal set of CWMP RPCs needed to fulfill the plan.
3. After each RPC response, updates `DeviceData` and re-evaluates.
4. Iterates until all declarations are satisfied.
The preset system (`applyPresets` in `cwmp.ts`) implements a policy engine:
presets are rules with precondition expressions that, when matched, contribute
provisions to the session. After provisions execute, if device data changed,
presets are re-evaluated (up to 4 cycles to prevent infinite loops).
### Provisions and Virtual Parameters
**Provisions** are the unit of configuration intent. Built-in provisions
(`default-provisions.ts`) include `refresh`, `value`, `tag`, `reboot`, `reset`,
`download`, and `instances`. Custom provisions are user-defined JavaScript
scripts stored in the `provisions` MongoDB collection.
**Virtual parameters** are scripts that present computed/derived values as if
they were real device parameters under the `VirtualParameters.*` namespace. They
run in two phases: a "get" phase reads real parameters and returns a computed
value; a "set" phase translates a desired value into real parameter changes.
Virtual parameters can reference other virtual parameters (up to depth 8).
### Sandbox (`lib/sandbox.ts`)
The sandbox provides a secure execution environment for provision and virtual
parameter scripts using `vm.Script` with a 50ms timeout. It uses a
**replay-based execution model**:
1. A script runs and calls `declare()` to request data.
2. When `commit()` is called, the script throws a sentinel symbol and exits.
3. The engine fetches the requested data via CWMP RPCs.
4. The script is **re-run from the beginning** with the fetched data available.
5. Earlier `declare()` calls return cached results; the script progresses
further.
6. This repeats until the script completes without throwing.
The sandbox API: `declare(path, timestamps, values)` returns a
`ParameterWrapper` proxy; `clear(path, timestamp, attributes)` invalidates
cached data; `ext(...args)` calls external extensions (results are cached per
revision to survive replays); `commit()` explicitly triggers a fetch cycle.
`Math.random()` is replaced with a seeded PRNG for determinism.
### Device Data Model (`lib/types.ts`, `lib/device.ts`)
`DeviceData` is the in-memory working copy of a device's parameter tree during a
session:
- `paths: PathSet` -- All known parameter paths.
- `timestamps: VersionedMap<Path, number>` -- When each path was last refreshed.
- `attributes: VersionedMap<Path, Attributes>` -- Per-path attributes (object,
writable, value, notification, accessList), each paired with a timestamp.
- `trackers` / `changes` -- Change tracking for re-evaluation.
`VersionedMap` (in `versioned-map.ts`) provides multi-revision snapshots,
enabling the sandbox replay model where scripts may be re-run at different
revision levels.
`device.ts` handles setting and clearing parameter data with invariant
enforcement (e.g., if `value` is set, `object` is forced to 0; parent paths are
ensured to exist). The `unpack()` function resolves wildcards and alias paths
against concrete device data.
### NBI (`lib/nbi.ts`)
A raw Node.js HTTP listener (no framework) with regex-based URL routing.
Endpoints include CRUD for presets, provisions, virtual parameters, objects, and
files; device task management (with optional synchronous execution via
connection request); fault management; generic collection querying; and ping.
The NBI uses MongoDB-style JSON queries directly. For the `devices` collection,
`query.ts` expands user-friendly queries by auto-appending `._value` to
parameter paths and generating multi-type interpretations (string, number, date,
regex) for filter values.
### File Server (`lib/fs.ts`)
A minimal HTTP file server reading from MongoDB GridFS. Supports GET/HEAD with
full HTTP caching (ETag, Last-Modified, If-None-Match, If-Modified-Since) and
Range requests (HTTP 206) for partial content downloads. Files are cached
in-memory via memoization.
### UI Backend (`lib/ui.ts`, `lib/ui/`)
A Koa application with JWT authentication, role-based authorization, and a rich
CRUD API under `/api/`. The root route serves an HTML shell that bootstraps the
Mithril.js SPA with injected config, user info, and hashed asset filenames.
`lib/ui/api.ts` defines generic CRUD endpoints for all resource types (devices,
presets, provisions, files, config, users, permissions, faults, tasks) with
authorization checks at every level. Specialized endpoints handle file
upload/download, synchronous task execution, tag management, password changes,
and CSV export.
`lib/ui/db.ts` translates between the UI's flat parameter representation and
MongoDB's nested document structure. The `flattenDevice()` function is the key
transformation: it recursively walks the nested device document and produces a
flat key-value map (e.g.,
`Device.WiFi.SSID.1.Name -> {value: [...], writable: true}`).
`lib/ui/local-cache.ts` caches permissions, users, and config in-process with
hash-based revision tracking. The `getConfig()` function resolves config
expressions with dynamic context and backward compatibility for legacy config
keys.
### Database Layer (`lib/db/`)
`db/db.ts` is the single entry point to MongoDB. It manages 14 collections:
| Collection | Purpose |
| ------------------- | -------------------------------- |
| `devices` | CPE device parameter trees |
| `presets` | Configuration rules |
| `provisions` | Provision scripts |
| `virtualParameters` | Virtual parameter scripts |
| `objects` | Generic objects |
| `tasks` | Queued device management tasks |
| `faults` | Error records with retry state |
| `operations` | In-flight async operations |
| `files` (GridFS) | Firmware images and config files |
| `permissions` | RBAC rules |
| `users` | User accounts |
| `config` | Key-value configuration |
| `cache` | Distributed cache (TTL index) |
| `locks` | Distributed locks (TTL index) |
`db/synth.ts` is the sophisticated expression-to-MongoDB query compiler. It
normalizes expressions, converts them to a Boolean satisfiability representation
using the `Clause` hierarchy, minimizes via Espresso, and emits MongoDB
`$and`/`$or`/`$not` filter objects. This is used by the UI backend.
`cwmp/db.ts` handles CWMP-specific persistence: loading the nested device
document into `DeviceData` (`fetchDevice`), diffing changes back into MongoDB
update operations (`saveDevice`), and managing faults, tasks, and operations.
### Query Systems
There are two separate query paths:
1. **NBI queries** (`lib/query.ts`) -- Processes MongoDB-style JSON queries from
external API consumers. Expands parameter names, generates multi-type value
interpretations, and passes through to MongoDB directly.
2. **UI/Expression queries** (`lib/db/synth.ts`) -- Compiles the internal
expression language into optimized MongoDB filters via Boolean minimization.
This is the more sophisticated path, used by the UI backend.
### Frontend (`ui/`)
A Mithril.js SPA with hash-based routing (`#!/`). Key modules:
- `app.ts` -- Route definitions. The `pagify()` function wraps each page into a
`RouteResolver` that handles initialization, error boundaries, and data
fulfillment.
- `layout.ts` -- Top-level layout (header, navigation, content, overlay).
- `store.ts` -- Centralized data store. Implements a query-based reactive cache
with deduplication and incremental fetching. The `fulfill()` method (called
after every render) batches pending queries, computes filter diffs via
`unionDiff()`, and fetches only missing data. Connection monitoring polls
every 3 seconds for server health, clock skew, and config changes.
- `components.ts` -- Component registry with a context propagation system. The
proxied `m()` function resolves string component names and the `m.context()`
API passes data (like the current device object) down the component tree
without explicit prop threading.
- `smart-query.ts` -- Translates user-friendly `Label: value` searches into
filter expressions with type-aware matching (string, number, timestamp, MAC
address, tag).
- `task-queue.ts` -- Two-stage task pipeline: staging (user configures tasks)
then queue (tasks are committed and executed via the backend).
- `dynamic-loader.ts` -- Lazy loading of heavy libraries (CodeMirror, YAML) via
dynamic `import()`.
### Build System (`build/`)
The build is a self-bootstrapping esbuild pipeline (`npm run build` pipes
`build/build.ts` through esbuild then node). It produces:
- **Backend binaries** -- 5 entry points bundled for Node.js 12+ with shebang
banners, executable permissions, and `.js` extension stripped.
- **Frontend bundle** -- `ui/app.ts` bundled for browsers (ESM, code-split) with
content-hashed filenames.
- **CSS** -- `ui/css/app.css` bundled and minified with content-hashed output.
- **SVG sprite** -- All icons in `ui/icons/` optimized via SVGO and combined
into a single sprite.
`build/assets.ts` is a compile-time bridge: at rest it contains placeholder
filenames; during build, the `assetsPlugin` replaces them with actual
content-hashed names so both backend and frontend reference the correct assets.
Build metadata includes the date and a hash derived from the git state, appended
to the package version.
## Cross-Cutting Concerns
### Multi-Level Caching
```
memoize.ts In-process function cache (2-4 min, two-generation rotation)
|
local-cache.ts In-process snapshot cache (5s refresh, hash-based revisions)
|
cache.ts MongoDB-backed distributed cache (configurable TTL)
|
MongoDB TTL index auto-expiration
```
Each service has its own `local-cache` that periodically checks the distributed
cache for staleness. Distributed locks (`lock.ts`) coordinate rebuilds so only
one worker does the expensive computation.
### Configuration (`lib/config.ts`)
Three-tier priority: CLI args > environment variables (`GENIEACS_*`) > config
file (`config.json`) > defaults. Supports per-device overrides by appending
`-OUI-ProductClass-SerialNumber` to option names.
### Distributed Locking (`lib/lock.ts`)
MongoDB-based mutual exclusion using upsert + duplicate key detection. TTL
indexes prevent deadlocks from crashed processes. Clock skew tolerance of 30
seconds.
### Authentication
- **CWMP devices** -- HTTP Basic or Digest auth, configurable per-device via
expressions (`auth.ts`).
- **UI users** -- JWT tokens in cookies. Passwords hashed with PBKDF2-SHA512
(10000 iterations). Role-based authorization via `Authorizer` with
expression-based filters and validators.
### Connection Requests (`lib/connection-request.ts`)
Three methods to ask a CPE device to initiate a CWMP session:
- **HTTP** -- Standard TR-069 connection request with Digest/Basic auth.
- **UDP** -- For NAT traversal (STUN-based) with HMAC-SHA1 signed messages.
- **XMPP** -- TR-069 Annex K via a full XMPP client (`xmpp-client.ts`).
### Extensions (`lib/extensions.ts`)
User-defined scripts executed in long-lived child processes (`genieacs-ext.ts`)
communicating via IPC. Processes are lazily spawned per script name and reused.
Each request gets a unique ID; responses are matched by ID with a configurable
timeout.
### Logging (`lib/logger.ts`)
Dual-stream structured logging (application + access logs). Supports simple and
JSON formats, systemd journal integration, and automatic log rotation detection.
Protocol traces can be written to a debug file in YAML or JSON format
(`debug.ts`).
### Three-Valued Logic
The system consistently implements SQL-style three-valued logic
(true/false/null). This is visible in expression evaluation, the `Clause`
hierarchy's separate true/false/null methods, and the 2-bit minterm encoding in
the Boolean minimizer. NULL means "unknown" and propagates through operations
following SQL semantics.
## Architectural Invariants
- **The expression system does not depend on any service-specific code.** The
`lib/common/expression/` modules are pure and shared across all services.
- **The sandbox is deterministic across replays.** `Math.random()` is seeded
from the device ID, `Date.now()` is controlled, and extension results are
cached. A script re-run with the same inputs produces the same outputs.
- **Services share no in-process state.** All cross-process coordination goes
through MongoDB (the `cache` and `locks` collections). Each worker process is
independent.
- **Device data is never mutated in place during a session.** `VersionedMap`
provides revision-based snapshots. The sandbox writes to new revisions;
earlier revisions remain readable for re-evaluation.
- **CWMP session exclusivity.** A distributed lock (`cwmp_session_<deviceId>`)
ensures only one session exists per device at a time. The lock is refreshed
periodically and released at session end.
- **The UI backend never queries MongoDB with raw user input.** All queries go
through the expression-to-MongoDB compiler (`db/synth.ts`), which validates
and normalizes expressions before generating filters.
CONTRIBUTING.md
+334 -21
View File
@@ -1,31 +1,348 @@
# Have a question?
# Contributing to GenieACS
Please ask in the [forum] instead of opening a Github Issue. The Github issue
tracker is strictly for bug reports.
## Questions and Support
# Issues
Please use the [forum](https://forum.genieacs.com) for questions, help requests,
and general discussion. GitHub Issues are reserved for confirmed bug reports.
We use GitHub issues to track bugs. Please ensure your bug description is clear
and has sufficient instructions to be able to reproduce the issue.
## Issues
If you face interoperability issues with your CPE, more often than not it's a
device related issue. Please consult the [forum] before submitting an Issue.
We use GitHub Issues to track bugs. When filing a bug report:
# Pull Requests
- Provide a clear description of the problem.
- Include steps to reproduce the issue.
- Note the GenieACS version, Node.js version, and MongoDB version.
- Include relevant log output if applicable.
Pull requests are welcome. If you have a fix for a bug, go ahead and create a
pull request. For other kinds of enhancements, please discuss in the [forum]
first.
If you face interoperability issues with a CPE, it is more often than not a
device-specific issue. Please consult the [forum](https://forum.genieacs.com)
before opening an Issue.
Please follow the guidelines [here][commit-guidelines] when committing your
changes.
## Pull Requests
Pull requests are welcome. For bug fixes, go ahead and open a PR directly. For
new features or significant changes, please discuss in the
[forum](https://forum.genieacs.com) first to ensure alignment with the project's
direction.
When submitting a PR:
- Keep changes focused. One PR should address one concern.
- Run `npm run lint` and `npm test` before submitting.
- Update documentation in `docs/` if your change affects user-facing behavior.
- Add tests where appropriate (see [Testing](#testing)).
- Write a clear PR description explaining what the change does and why.
## Code Style
### Naming Conventions
- **Variables and functions**: `camelCase`
- **Classes**: `PascalCase`
- **Interfaces and type aliases**: `PascalCase`, without an `I` or `T` prefix
- **Module-level constants**: `UPPER_SNAKE_CASE`
- **Private class members**: prefixed with underscore (`_name`, `_cache`)
Choose descriptive, meaningful names. A longer clear name is better than a short
ambiguous one.
### Imports
Organize imports in three groups, in this order:
1. Node.js built-in modules, using the `node:` prefix
2. External packages
3. Local project modules
Use `.ts` extensions for all local imports.
```typescript
import { readFileSync } from "node:fs";
import * as http from "node:http";
import { Collection } from "mongodb";
import { connect, disconnect } from "./db.ts";
import Path from "./common/path.ts";
```
### Functions and Exports
Use the `function` keyword for all named and exported function declarations.
Arrow functions should only be used for callbacks and short inline expressions.
```typescript
// Named/exported functions use function keyword
export function processRequest(req: Request): Response {
// ...
}
// Arrow functions for callbacks
items.filter((item) => item.active);
promise.then((result) => {
// ...
});
```
### TypeScript
- All function declarations must have explicit return types. This is enforced by
ESLint.
- Using `any` is permitted where full typing would be impractical, but prefer
more specific types when reasonable.
- Prefer `Record<string, T>` over `{ [key: string]: T }` for mapped types.
- Use type assertions (`as`) sparingly and only when you have stronger type
knowledge than the compiler.
### Comments
Code should be self-documenting. Use comments sparingly and only when the "why"
is not obvious from the code itself. When you do comment, use inline `//` style.
Do not use JSDoc (`/** */`) or block comments (`/* */`).
```typescript
// Good: explains a non-obvious reason
// Escapes everything except alphanumerics and underscore
function escapeRegExp(str: string): string {
/* ... */
}
// Good: links to an external reference
// Source: http://stackoverflow.com/a/6969486
// Good: flags a known limitation
// TODO support "MD5-sess" algorithm directive
// Bad: restates what the code does
// Loop through each item and increment the counter
for (const item of items) {
counter++;
}
```
### Error Handling
Use `== null` (not `=== null || === undefined`) for null/undefined checks.
ESLint is configured to allow this.
## Testing
### When to Write Tests
Tests are most valuable for pure logic: parsers, data transformations,
algorithms, and utility functions where inputs and outputs are well defined. A
regression test accompanying a bug fix is also worthwhile to prevent the same
issue from resurfacing.
Not every change needs a test. Use judgment and consider the likelihood and cost
of breakage. Avoid writing tests that duplicate coverage already provided by
existing tests, and resist the urge to test trivial code just for the sake of
coverage numbers.
### Conventions
Tests use the Node.js built-in test runner (`node:test`) and assertion module
(`node:assert`). No external test libraries or mocking frameworks are used.
```typescript
import test from "node:test";
import assert from "node:assert";
void test("parseValue returns correct type for integer strings", () => {
const result = parseValue("42");
assert.strictEqual(result, 42);
});
void test("parseValue throws on invalid input", () => {
assert.throws(() => parseValue(""), new Error("empty value"));
});
```
Key conventions:
- Prefix `test()` calls with `void` to satisfy the `no-floating-promises` lint
rule.
- Keep tests flat. Do not nest `describe` blocks.
- Use descriptive test names that state what is being tested and the expected
outcome.
- Use `assert.strictEqual()` for value comparisons and
`assert.deepStrictEqual()` for objects and arrays.
Test files live in the `test/` directory and are named after the module they
test (e.g. `test/path.ts` tests `lib/common/path.ts`).
## Dependencies
This project deliberately keeps its dependency footprint small. Before adding a
new dependency:
- Prefer Node.js built-in APIs when they can do the job.
- Consider whether the functionality is simple enough to implement directly.
- Justify the addition in your PR description.
Do not add development tool dependencies (linter plugins, editor integrations,
etc.) without prior discussion.
## File Organization
| Directory | Contents |
| ------------- | -------------------------------------------- |
| `lib/` | Core server-side application code |
| `lib/common/` | Code shared between server and browser |
| `lib/db/` | Database layer (MongoDB) |
| `lib/ui/` | UI backend helpers |
| `ui/` | Frontend code (Mithril.js SPA) |
| `bin/` | Service entry points |
| `build/` | Build tooling |
| `test/` | Test files |
| `docs/` | User documentation (Sphinx/reStructuredText) |
| `public/` | Static assets (favicon, logo) |
Place new code in the directory that matches its purpose. Server-side logic
belongs in `lib/`, code that must run in both Node.js and the browser belongs in
`lib/common/`, and frontend-only code belongs in `ui/`.
## Documentation
User documentation lives in the `docs/` directory as reStructuredText files
built with Sphinx and published to
[docs.genieacs.com](https://docs.genieacs.com).
When your change affects user-facing behavior:
- Update the relevant documentation in `docs/`.
- If adding a new feature, consider whether it warrants a new page or a section
in an existing page.
- Keep documentation concise and practical. Match the existing tone: direct,
factual, no filler.
Documentation changes should be included in the same PR as the code change, not
submitted separately.
### ARCHITECTURE.md
`ARCHITECTURE.md` describes the high-level architecture of the project: the
service boundaries, major subsystems, data flow, and key invariants. It is aimed
at contributors who need a mental map of the codebase.
This file has a different update cadence than `docs/`. It should be revisited a
few times a year rather than kept in lockstep with every code change. When you
do update it, follow these principles:
- **Only describe things that are unlikely to change frequently.** Module
responsibilities, service boundaries, key data structures, and architectural
invariants belong here. Implementation details, function signatures, and
config option lists do not.
- **Name important files, modules, and types but do not link them.** Links go
stale. Encourage the reader to use symbol search to find named entities; this
also helps them discover related, similarly named things.
- **Keep it short.** Every recurring contributor will read it. A shorter
document is less likely to become stale and more likely to be maintained.
- **Describe the "what" and "where", not the "how".** This is a map of the
country, not an atlas of its states. Pull detailed explanations into inline
code comments or separate documents.
- **Call out architectural invariants explicitly.** Important invariants are
often expressed as the _absence_ of something (e.g., "the expression system
does not depend on service-specific code") and are hard to discover by reading
code alone.
## Git Workflow
### Commit Messages
This project follows the
[Conventional Commits](https://www.conventionalcommits.org/) format:
<type>(<scope>): <subject>
[optional body]
- Use the imperative mood, present tense: "Fix bug", not "Fixed bug" or "Fixes
bug".
- Do not capitalize the first letter of the subject (the type prefix handles
visual structure).
- Do not end the subject line with a period.
- Keep the subject line under 72 characters.
- When more context is helpful, add a body separated from the subject by a blank
line, wrapped at 72 characters. The goal is to provide enough information for
someone scanning the commit history to find a specific change (e.g. for
troubleshooting or rebasing) or to draft changelog entries for a release.
Don't be verbose, but don't be cryptic either.
#### Types
| Type | When to use |
| ---------- | --------------------------------------------------------------------------- |
| `fix` | Bug fixes |
| `feat` | New features or capabilities |
| `refactor` | Code restructuring with no behavior change |
| `test` | Adding or updating tests |
| `docs` | Documentation changes (`docs/`, `CONTRIBUTING.md`, `README.md`) |
| `build` | Build system, dependencies, esbuild config, `package.json` scripts |
| `chore` | Maintenance tasks that don't fit above (`.gitignore`, tooling config, etc.) |
#### Scopes
Scope is optional. Use it when it adds useful context; omit it when the change
is cross-cutting or the subject already makes it obvious.
| Scope | Covers |
| ------ | ----------------------------------- |
| `cwmp` | CWMP service (including extensions) |
| `nbi` | Northbound REST API service |
| `fs` | File service |
| `ui` | Web UI (frontend and backend) |
| `db` | Database layer |
If a change touches multiple scopes, either pick the primary one or omit the
scope entirely.
#### Examples
feat(nbi): add bulk device delete endpoint
fix(cwmp): handle missing ParameterKey in InformResponse
refactor(db): replace raw queries with parameterized calls
fix(ui): correct parameter table sort order
test: add XML parser edge case coverage
docs: update provisioning guide for new API
build: upgrade esbuild to v0.20
fix(cwmp): increase server timeout to 2 mins
To allow enough time for running unindexed queries in large
deployments.
### Branches
- `master` is the main development branch.
- Create a feature or fix branch for your work and open a PR against `master`.
- Use concise, descriptive branch names in lowercase with hyphens:
`fix-race-condition`, `support-xmpp-requests`.
## Changelog
The changelog (`CHANGELOG.md`) is maintained for each release and is written for
users and system administrators, not developers.
- Write entries as clear, user-facing prose. Do not copy commit messages
verbatim.
- Each entry should describe what changed and, when helpful, why it matters.
- Group entries under a version heading with the release date:
`## 1.2.13 (2024-06-06)`.
- Start each entry with a verb: "Fix", "Add", "Improve", "Remove", etc.
- Include enough context that a user can understand the impact without reading
the code.
You do not need to update the changelog in your PR. The maintainer will add
changelog entries during the release process.
## Contributor License Agreement
By submitting a pull request to this repository, you acknowledge that, while
maintaining copyright, you grant Zaid Abdulla a perpetual, worldwide, non-
exclusive, no-charge, royalty-free, irrevocable license to reproduce, prepare
derivative works of, publicly display, publicly perform, sublicense, and
maintaining copyright, you grant GenieACS Inc. a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable license to reproduce,
prepare derivative works of, publicly display, publicly perform, sublicense, and
distribute your contributions and such derivative works under the AGPLv3 license
or any other license terms, including, but not limited to, proprietary or
commercial license terms.
@@ -33,7 +350,3 @@ commercial license terms.
You confirm that you own or have rights to distribute and sublicense the source
code contained therein, and that your content does not infringe upon the
intellectual property rights of a third party.
[commit-guidelines]:
http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
[forum]: https://forum.genieacs.com