1.0.5

Use a valid SPDX license expression
Merge pull request #110 from shweelan/v1.0
2015-11-22 01:46:35 +03:00 · 2015-11-07 20:35:10 +03:00 · 2015-09-26 21:57:19 +03:00 · 2015-09-07 14:11:20 +03:00 · 2015-07-09 22:22:04 +03:00 · 2015-07-05 03:44:44 +03:00
236 changed files with 6086 additions and 49143 deletions
@@ -1,28 +0,0 @@
 # Dependências e artefatos de build
 node_modules/
 dist/
 # Arquivos de ambiente (contêm segredos)
 .env
 # Logs e debug
 logs/
 *.log
 # Artefatos de teste
 test/*.js
 # Docs geradas
 docs/_build/
 # Dados de desenvolvimento
 .git/
 .github/
 # IDEs
 .vscode/
 .idea/
 # OS files
 .DS_Store
 Thumbs.db
@@ -1,69 +0,0 @@
 # ============================================================
 # GenieACS — variáveis de ambiente
 # Copie este arquivo para .env e ajuste os valores
 # NUNCA faça commit do .env com dados reais em repositórios
 # públicos.
 # ============================================================
 # ── MongoDB ─────────────────────────────────────────────────
 # Quando usando docker-compose o host é o nome do serviço "mongo"
 GENIEACS_MONGODB_CONNECTION_URL=mongodb://mongo:27017/genieacs
 # ── Extensões ───────────────────────────────────────────────
 GENIEACS_EXT_DIR=/opt/genieacs/config/ext
 GENIEACS_EXT_TIMEOUT=3000
 # ── Logs ────────────────────────────────────────────────────
 # json | simple
 GENIEACS_LOG_FORMAT=json
 GENIEACS_ACCESS_LOG_FORMAT=json
 # Para gravar logs em arquivo (opcional, senão vai para stdout/stderr)
 # GENIEACS_CWMP_LOG_FILE=/opt/genieacs/logs/cwmp.log
 # GENIEACS_CWMP_ACCESS_LOG_FILE=/opt/genieacs/logs/cwmp-access.log
 # GENIEACS_NBI_LOG_FILE=/opt/genieacs/logs/nbi.log
 # GENIEACS_NBI_ACCESS_LOG_FILE=/opt/genieacs/logs/nbi-access.log
 # GENIEACS_FS_LOG_FILE=/opt/genieacs/logs/fs.log
 # GENIEACS_FS_ACCESS_LOG_FILE=/opt/genieacs/logs/fs-access.log
 # GENIEACS_UI_LOG_FILE=/opt/genieacs/logs/ui.log
 # GENIEACS_UI_ACCESS_LOG_FILE=/opt/genieacs/logs/ui-access.log
 # ── Debug (opcional) ────────────────────────────────────────
 # GENIEACS_DEBUG_FILE=/opt/genieacs/logs/debug.yaml
 # GENIEACS_DEBUG_FORMAT=yaml
 # ── Worker processes (0 = 1 por CPU core) ───────────────────
 GENIEACS_CWMP_WORKER_PROCESSES=1
 GENIEACS_NBI_WORKER_PROCESSES=1
 GENIEACS_FS_WORKER_PROCESSES=1
 GENIEACS_UI_WORKER_PROCESSES=1
 # ── Portas ──────────────────────────────────────────────────
 GENIEACS_CWMP_PORT=7547
 GENIEACS_NBI_PORT=7557
 GENIEACS_FS_PORT=7567
 GENIEACS_UI_PORT=3000
 # ── Interface de bind (:: = todas as interfaces) ────────────
 GENIEACS_CWMP_INTERFACE=::
 GENIEACS_NBI_INTERFACE=::
 GENIEACS_FS_INTERFACE=::
 GENIEACS_UI_INTERFACE=::
 # ── UI — JWT Secret (OBRIGATÓRIO trocar em produção!) ───────
 # Mínimo 32 chars, máximo 64 chars
 GENIEACS_UI_JWT_SECRET=mude-isto-em-producao-use-uma-string-aleatoria-longa
 # ── FS — URL prefix (configure se CWMP e FS estiverem em ────
 #         hosts/IPs diferentes ou atrás de proxy)            
 # GENIEACS_FS_URL_PREFIX=http://seu-ip-publico:7567/
 # ── SSL (opcional — passe o caminho para cert/key PEM) ──────
 # GENIEACS_CWMP_SSL_CERT=/opt/genieacs/certs/cwmp.crt
 # GENIEACS_CWMP_SSL_KEY=/opt/genieacs/certs/cwmp.key
 # GENIEACS_NBI_SSL_CERT=/opt/genieacs/certs/nbi.crt
 # GENIEACS_NBI_SSL_KEY=/opt/genieacs/certs/nbi.key
 # GENIEACS_FS_SSL_CERT=/opt/genieacs/certs/fs.crt
 # GENIEACS_FS_SSL_KEY=/opt/genieacs/certs/fs.key
 # GENIEACS_UI_SSL_CERT=/opt/genieacs/certs/ui.crt
 # GENIEACS_UI_SSL_KEY=/opt/genieacs/certs/ui.key
@@ -1,50 +0,0 @@
 GitHub Copilot Instructions: GenieACS -> UniFi UI Theme
 You are an expert senior software engineer and frontend architect focused on UI/UX modernization, maintainability, and clean architecture. Your primary goal is to help modify the GenieACS user interface to closely resemble the UniFi Controller, while strictly ensuring the codebase remains easy to synchronize (via merge or rebase) with the original GenieACS upstream repository for bug fixes and core updates.
 1. Core Principles (Non-Negotiable)
 Sync-Friendly Architecture (Crucial): Write code defensively to prevent future merge conflicts with the original GenieACS project. Prefer extending, overriding, or wrapping components rather than heavily modifying deeply ingrained original core files.
 Visual Fidelity (UniFi): Implement the clean, dashboard-centric, modern aesthetic of the UniFi Controller. Use comparable color palettes, typography, spacing, and network topology layouts.
 Safety First: Never modify the underlying TR-069 communication logic, backend models, or database schemas unless explicitly requested. The focus is strictly on the presentation layer (View).
 Minimal Footprint: When generating files, aim to isolate custom UI assets. Keep custom CSS/SCSS and new UI components in distinct directories or files whenever possible.
 2. GenieACS Customization & Upstream Integration
 CRITICAL: Whenever the user requests a modification to a page or component, you MUST prioritize the following strategies to keep the fork maintainable:
 Style Overrides: Prefer creating custom stylesheets (e.g., unifi-theme.scss) to override existing GenieACS classes instead of changing inline styles or the original CSS files directly.
 Component Wrapping: If an existing UI component needs significant structural changes, create a new "UniFi-styled" wrapper component that consumes the original GenieACS state/props, rather than gutting the original component.
 Clear Demarcation: When modifying an original GenieACS file is unavoidable, clearly mark the beginning and end of the custom code with comments (e.g., // --- BEGIN UNIFI THEME --- and // --- END UNIFI THEME ---). This makes resolving Git merge conflicts significantly easier.
 3. Code Generation Rules
 A. Refactoring & Modification
 Context Awareness: Read the surrounding code to understand the existing state management and API calls. Do not break the data flow between the GenieACS backend and the frontend.
 Preserve Logic: When changing how data is displayed (e.g., changing a simple table to a UniFi-style card grid), ensure all original data points remain accessible or are intentionally hidden via CSS, rather than deleting the data-fetching logic.
 B. New Features (UI Components)
 Modularity: Build reusable UI components (Buttons, Cards, Navbars) that match the UniFi design system, so they can be applied globally across the GenieACS interface.
 Responsiveness: Ensure all new UI elements are fully responsive, as network management dashboards are often viewed on various screen sizes.
 C. Testing
 Prioritize visual regression checks and ensure that frontend unit tests (if applicable) are updated to reflect structural DOM changes. Ensure the connection to the GenieACS API remains mocked and functional in tests.
 4. Response Format
 Plan First: Before generating code, briefly outline your strategy. Specifically state which files will be modified and how you plan to implement the change while minimizing upstream merge conflicts.
 Complete Code: Output the full, functional block of code needed for the file. Do not use placeholders like // ... rest of code unless the file is massive and the change is highly localized.
 Git Context: If a change requires creating a new file to override an old one, explain how to integrate it into the project's build system.
 5. "Do No Harm" Checklist
 Before outputting the final response, ask yourself:
 Does this modification break the core GenieACS device management logic?
 Will this change cause a massive merge conflict if the upstream repository updates this specific component? (If yes, rethink the approach).
 Does the resulting UI genuinely reflect the UniFi Controller design language?
@@ -1,5 +1,4 @@
 *~
 *.js
 node_modules
-dist
+debug/*
 docs/_build
 .env
@@ -0,0 +1,14 @@
 *~
 .npmignore
 config/*
 !config/*-sample.*
 !config/custom_commands
 debug/*
 !debug/.gitkeep
 lib/*.coffee
 tools/*.coffee
 test/*.coffee
@@ -1,2 +0,0 @@
 dist
 npm-shrinkwrap.json
@@ -1,82 +0,0 @@
 # AGENTS.md — GenieACS
 GenieACS is a TR-069 Auto Configuration Server for remote management of CPE
 devices (routers, modems, gateways). TypeScript codebase compiled with esbuild,
 backed by MongoDB.
 ## Architecture Overview
 Four services share a single MongoDB instance:
 - **CWMP** (port 7547) — TR-069 protocol handler; manages device sessions
 - **NBI** (port 7557) — Northbound REST API for external consumers
 - **FS** (port 7567) — File server for firmware/config (GridFS-backed)
 - **UI** (port 3000) — Web interface (Koa backend + Mithril.js SPA frontend)
 Key subsystems: expression engine (`lib/common/expression/`) compiles a
 Lisp-like DSL used for queries, config, and authorization; session engine
 (`lib/session.ts`) drives CWMP interactions via declarations rather than
 imperative RPCs; sandbox (`lib/sandbox.ts`) runs user-defined provision scripts
 in `vm.Script` with deterministic replay.
 Read `ARCHITECTURE.md` for a full map of the codebase when working on unfamiliar
 areas. It covers service boundaries, the expression pipeline, the path system,
 the CWMP session state machine, the database layer, and architectural
 invariants.
 ## Project Structure
 - `lib/` — Core server-side logic
 - `lib/common/` — Shared code (runs in both Node.js and browser)
 - `lib/db/` — MongoDB database layer
 - `lib/ui/` — UI backend helpers
 - `ui/` — Frontend SPA (Mithril.js)
 - `bin/` — Service entry points (5 executables)
 - `build/` — Build scripts (esbuild pipeline)
 - `test/` — Unit tests (node:test)
 - `docs/` — User docs (Sphinx/reStructuredText)
 - `public/` — Static assets (favicon, logo)
 ## Build / Lint / Test Commands
 ```bash
 npm run build # Production build (esbuild pipeline -> dist/)
 NODE_ENV=development npm run build # Dev build (no minification)
 npm run lint # Prettier + ESLint + tsc --noEmit in parallel
 npm test # Compile tests with esbuild, run with node --test
 ```
 ### Running a Single Test File
 ```bash
 esbuild --log-level=warning --bundle --platform=node --target=node18 \
  --packages=external --sourcemap=inline --outdir=test test/path.ts \
  && node --test --enable-source-maps test/path.js \
  && rm test/path.js
 ```
 ### Running a Single Test Case
 ```bash
 esbuild --log-level=warning --bundle --platform=node --target=node18 \
  --packages=external --sourcemap=inline --outdir=test test/path.ts \
  && node --test --enable-source-maps --test-name-pattern="^parse$" test/path.js \
  && rm test/path.js
 ```
 ### Lint Sub-commands
 ```bash
 prettier --prose-wrap always --write .
 eslint 'bin/*.ts' 'lib/**/*.ts' 'ui/**/*.ts' 'test/**/*.ts' 'build/**/*.ts'
 tsc --noEmit
 ```
 ## Before Committing
 Read `CONTRIBUTING.md` and ensure your changes comply with it. In particular:
 - Run `npm run lint` and `npm test` and fix any failures.
 - Follow the code style, naming, import, and comment conventions documented
  there.
 - Use the Conventional Commits format for commit messages.
@@ -1,479 +0,0 @@
 # Architecture
 This document describes the high-level architecture of GenieACS. If you want to
 familiarize yourself with the codebase, you are in the right place.
 ## Bird's Eye View
 GenieACS is a TR-069 Auto Configuration Server (ACS). It manages CPE devices
 (routers, modems, gateways) using the CWMP protocol (TR-069). The system
 consists of four network-facing services that share a MongoDB database:
 ```
                  +------------+
  CPE Devices --->| CWMP (7547)|-+
                  +------------+ |
                  +-----------+  |     +---------+
  CPE Devices --->| FS (7567) |--+---->| MongoDB |
                  +-----------+  |     +---------+
                  +-----------+  |
  OSS / Scripts ->| NBI (7557)|--+
                  +-----------+  |
                  +-----------+  |
  Administrators->| UI (3000) |--+
                  +-----------+
 ```
 - **CWMP** -- The core TR-069 protocol server. CPE devices connect here to
  report their state and receive configuration instructions via SOAP/XML over
  HTTP.
 - **NBI** -- Northbound Interface. A REST API for external systems (OSS/BSS,
  scripts, automation) to manage devices, tasks, presets, and configuration
  programmatically.
 - **FS** -- File Server. Serves firmware images and configuration files to CPE
  devices during download operations.
 - **UI** -- Web interface. A Koa-based backend serving a Mithril.js single-page
  application for administrators to browse devices, manage configuration, and
  trigger operations.
 All four services follow the same process model: a primary process forks
 configurable worker processes via Node.js `cluster` (see `cluster.ts`). Workers
 connect to MongoDB and start an HTTP(S) server.
 ## Code Map
 ```
 bin/                        Service entry points (5 executables)
 lib/                        All backend logic
  common/                   Shared utilities (expression engine, Path, errors)
    expression.ts           Expression class hierarchy (base + subclasses)
    expression/             Expression parser, evaluator, normalizer, minimizer
  cwmp/                     CWMP-service-specific DB and caching
  db/                       Database layer (MongoDB collections, query synthesis)
  ui/                       UI-service-specific API, DB, and caching
  types/                    (empty, reserved)
 ui/                         Frontend SPA (Mithril.js)
  components/               Reusable UI components (parameter, tags, ping, etc.)
  css/                      Stylesheets (vanilla CSS)
  icons/                    SVG icons (compiled into a sprite)
 test/                       Unit tests (Node.js native test runner)
 build/                      Build scripts (esbuild-based)
 public/                     Static assets (logo, favicon)
 ```
 ### Entry Points (`bin/`)
 Each file in `bin/` bootstraps one service. They are structurally identical:
 initialize logging, read config, fork workers in the primary process, connect to
 MongoDB and start the HTTP server in each worker.
 - `genieacs-cwmp.ts` -- CWMP service. Unique in that it disables HTTP keep-alive
  (`keepAliveTimeout: 0`) and provides custom `onConnection` / `onClientError`
  hooks for TR-069 session lifecycle management.
 - `genieacs-nbi.ts` -- NBI service. Straightforward REST API server.
 - `genieacs-fs.ts` -- File server. The leanest service; does not use the
  extensions subsystem.
 - `genieacs-ui.ts` -- UI service. Wraps the Koa application as the HTTP
  listener.
 - `genieacs-ext.ts` -- **Not a service.** This is a child process worker spawned
  by the extensions subsystem. It communicates with its parent via IPC,
  executing user-defined extension scripts in isolation.
 ### The Expression System (`lib/common/expression.ts`, `lib/common/expression/`)
 The `Expression` class (defined in `lib/common/expression.ts`) is the most
 important abstraction in the codebase. It uses a typed class hierarchy:
 `Expression.Literal` (wraps `string | number | boolean | null`),
 `Expression.Parameter` (wraps a `Path`), `Expression.Binary` (operator +
 left/right), `Expression.Unary` (operator + operand), `Expression.FunctionCall`
 (name + args), and `Expression.Conditional` (condition/then/otherwise). For
 example, in the SQL-like text syntax:
 ```
 Device.ModelName = "BrandX" AND Events.Inform > 1000
 ```
 Expressions are used pervasively:
 - **Query/filter language** -- Database queries for devices, faults, presets,
  etc. are all represented as expressions and compiled to MongoDB filters.
 - **Configuration values** -- Config entries and preset preconditions are
  expressions, enabling dynamic evaluation.
 - **Authorization** -- Permission filters and validators are expressions.
 - **Pagination cursors** -- Keyset pagination boundaries are expressed as filter
  expressions.
 The expression pipeline flows through four modules:
 1. `parser.ts` -- Parses a SQL-like text syntax into the AST using a hand-rolled
   recursive descent parser with a `Cursor`-based scanner. Also provides
   `stringifyExpression()` for serialization. The `map()` / `mapAsync()`
   tree-walking primitives are abstract methods on the `Expression` class, and
   `stringify()` is now `Expression.toString()`.
 2. `normalize.ts` -- Algebraic normalization using exact rational polynomial
   arithmetic over native `bigint` values. The `Polynomial` class extends
   `Expression` as an intermediate representation. Ensures equivalent
   expressions have the same canonical form (e.g., `a + 2 > b` and `a > b - 2`
   normalize identically).
 3. `synth.ts` -- Boolean logic minimization via the Espresso algorithm
   (espresso-iisojs). Converts expressions to minimal sum-of-products form with
   three-valued logic (true/false/null). Handles domain-specific constraints
   like comparison ordering and LIKE pattern relationships.
 4. `evaluate.ts` -- Runtime evaluation. The `reduce()` function evaluates
   operators on literal values and supports partial evaluation (returns a
   reduced expression if some values are unknown). Parameter resolution is
   handled by the `Expression.evaluate()` method (in `lib/common/expression.ts`)
   which calls `reduce()` after mapping children via a user-supplied callback.
 `pagination.ts` implements cursor-based pagination by generating filter
 expressions from sort-key bookmarks.
 ### The Path System (`lib/common/path.ts`, `lib/common/path-set.ts`)
 `Path` represents a TR-069 parameter path (e.g., `Device.WiFi.SSID.1.Name`).
 Paths can contain wildcards (`*`) and alias expressions (`[key:value]`) for
 query-based addressing.
 Key design decisions:
 - **Cached** -- `Path.parse()` caches instances in a two-generation LRU cache
  rotated every 120 seconds. The constructor is public (used directly by the
  parser and by methods like `slice()`, `concat()`, and `stripAlias()`).
 - **Bitmask encoding** -- The `wildcard` and `alias` fields are bitfields for
  O(1) segment-type checking. This limits paths to 32 segments. A `colon` field
  tracks the number of attribute path segments (after a `:` separator), enabling
  `paramLength` and `attrLength` accessors.
 - **Immutable** -- Segment arrays are `Object.freeze()`-d.
 `PathSet` is a multi-indexed collection of paths supporting pattern-matching
 queries. It maintains separate `paramSegmentIndex` and `attrSegmentIndex` arrays
 (one `Map<string, Set<Path>>` per position), plus a `stringIndex` map. The
 `find()` method takes bitmasks to control which segments require exact matches
 vs. wildcard compatibility, then uses set intersection across the smallest index
 sets. A higher-level `findCompat()` method computes the appropriate bitmasks for
 superset/subset matching.
 ### CWMP Protocol Layer (`lib/cwmp.ts`, `lib/soap.ts`, `lib/xml-parser.ts`)
 `xml-parser.ts` is a custom single-pass XML parser (no DOM). It scans
 character-by-character with bitwise state flags, building a tree of elements
 with namespace support. Does not support CDATA.
 `soap.ts` handles both parsing CPE SOAP messages and generating ACS SOAP
 responses. It dispatches on the SOAP body's method name to type-specific parsers
 for Inform, TransferComplete, GetParameterNamesResponse, etc. Supports CWMP
 versions 1.0 through 1.4.
 `cwmp.ts` is the HTTP-level CWMP request handler. It manages the session state
 machine:
 1. **State 0** -- Expects an Inform. Authenticates the device (Basic or Digest
   auth, configurable via expression). Acquires a distributed lock. Loads device
   data from MongoDB. Sends InformResponse.
 2. **State 1** -- Waits for the CPE to send an empty POST (ready for ACS RPCs).
   Processes any TransferComplete messages.
 3. **State 2** -- The ACS drives RPCs (GetParameterNames, GetParameterValues,
   SetParameterValues, AddObject, DeleteObject, Download, Reboot, FactoryReset).
   The CPE responds to each.
 Session persistence across TCP disconnects: when a socket closes mid-session,
 the entire `SessionContext` is serialized to Redis (the MongoDB `cache`
 collection) and restored when the CPE reconnects (identified by a session
 cookie).
 ### Session Engine (`lib/session.ts`)
 The session engine implements the **declaration-driven data fetching** pattern.
 Rather than issuing RPCs imperatively, provisions create `Declaration` objects
 stating what paths and attributes they need to read or write. The engine then:
 1. Processes all declarations into a `SyncState` -- a structured plan of which
   parameters to refresh, which values to set, which instances to create/delete,
   etc.
 2. Generates the minimal set of CWMP RPCs needed to fulfill the plan.
 3. After each RPC response, updates `DeviceData` and re-evaluates.
 4. Iterates until all declarations are satisfied.
 The preset system (`applyPresets` in `cwmp.ts`) implements a policy engine:
 presets are rules with precondition expressions that, when matched, contribute
 provisions to the session. After provisions execute, if device data changed,
 presets are re-evaluated (up to 4 cycles to prevent infinite loops).
 ### Provisions and Virtual Parameters
 **Provisions** are the unit of configuration intent. Built-in provisions
 (`default-provisions.ts`) include `refresh`, `value`, `tag`, `reboot`, `reset`,
 `download`, and `instances`. Custom provisions are user-defined JavaScript
 scripts stored in the `provisions` MongoDB collection.
 **Virtual parameters** are scripts that present computed/derived values as if
 they were real device parameters under the `VirtualParameters.*` namespace. They
 run in two phases: a "get" phase reads real parameters and returns a computed
 value; a "set" phase translates a desired value into real parameter changes.
 Virtual parameters can reference other virtual parameters (up to depth 8).
 ### Sandbox (`lib/sandbox.ts`)
 The sandbox provides a secure execution environment for provision and virtual
 parameter scripts using `vm.Script` with a 50ms timeout. It uses a
 **replay-based execution model**:
 1. A script runs and calls `declare()` to request data.
 2. When `commit()` is called, the script throws a sentinel symbol and exits.
 3. The engine fetches the requested data via CWMP RPCs.
 4. The script is **re-run from the beginning** with the fetched data available.
 5. Earlier `declare()` calls return cached results; the script progresses
   further.
 6. This repeats until the script completes without throwing.
 The sandbox API: `declare(path, timestamps, values)` returns a
 `ParameterWrapper` proxy; `clear(path, timestamp, attributes)` invalidates
 cached data; `ext(...args)` calls external extensions (results are cached per
 revision to survive replays); `commit()` explicitly triggers a fetch cycle.
 `Math.random()` is replaced with a seeded PRNG for determinism.
 ### Device Data Model (`lib/types.ts`, `lib/device.ts`)
 `DeviceData` is the in-memory working copy of a device's parameter tree during a
 session:
 - `paths: PathSet` -- All known parameter paths.
 - `timestamps: VersionedMap<Path, number>` -- When each path was last refreshed.
 - `attributes: VersionedMap<Path, Attributes>` -- Per-path attributes (object,
  writable, value, notification, accessList), each paired with a timestamp.
 - `trackers` / `changes` -- Change tracking for re-evaluation.
 `VersionedMap` (in `versioned-map.ts`) provides multi-revision snapshots,
 enabling the sandbox replay model where scripts may be re-run at different
 revision levels.
 `device.ts` handles setting and clearing parameter data with invariant
 enforcement (e.g., if `value` is set, `object` is forced to 0; parent paths are
 ensured to exist). The `unpack()` function resolves wildcards and alias paths
 against concrete device data.
 ### NBI (`lib/nbi.ts`)
 A raw Node.js HTTP listener (no framework) with regex-based URL routing.
 Endpoints include CRUD for presets, provisions, virtual parameters, objects, and
 files; device task management (with optional synchronous execution via
 connection request); fault management; generic collection querying; and ping.
 The NBI uses MongoDB-style JSON queries directly. For the `devices` collection,
 `query.ts` expands user-friendly queries by auto-appending `._value` to
 parameter paths and generating multi-type interpretations (string, number, date,
 regex) for filter values.
 ### File Server (`lib/fs.ts`)
 A minimal HTTP file server reading from MongoDB GridFS. Supports GET/HEAD with
 full HTTP caching (ETag, Last-Modified, If-None-Match, If-Modified-Since) and
 Range requests (HTTP 206) for partial content downloads. Files are cached
 in-memory via memoization.
 ### UI Backend (`lib/ui.ts`, `lib/ui/`)
 A Koa application with JWT authentication, role-based authorization, and a rich
 CRUD API under `/api/`. The root route serves an HTML shell that bootstraps the
 Mithril.js SPA with injected config, user info, and hashed asset filenames.
 `lib/ui/api.ts` defines generic CRUD endpoints for all resource types (devices,
 presets, provisions, files, config, users, permissions, faults, tasks) with
 authorization checks at every level. Specialized endpoints handle file
 upload/download, synchronous task execution, tag management, password changes,
 and CSV export.
 `lib/ui/db.ts` translates between the UI's flat parameter representation and
 MongoDB's nested document structure. The `flattenDevice()` function is the key
 transformation: it recursively walks the nested device document and produces a
 flat key-value map with colon-delimited attribute keys (e.g.,
 `"Device.WiFi.SSID.1.Name" -> value`,
 `"Device.WiFi.SSID.1.Name:type" -> "xsd:string"`,
 `"Device.WiFi.SSID.1.Name:writable" -> true`). The `FlatDevice` type is
 `Record<string, Value>` where `Value = string | number | boolean | null`.
 `lib/ui/local-cache.ts` caches permissions, users, and config in-process with
 hash-based revision tracking. The `getConfig()` function uses typed overloads --
 it takes a typed default value (`string`, `number`, or `boolean`) and an
 expression evaluation callback, returning a value of the same type as the
 default.
 ### Database Layer (`lib/db/`)
 `db/db.ts` is the single entry point to MongoDB. It manages 14 collections:
 | Collection          | Purpose                          |
 | ------------------- | -------------------------------- |
 | `devices`           | CPE device parameter trees       |
 | `presets`           | Configuration rules              |
 | `provisions`        | Provision scripts                |
 | `virtualParameters` | Virtual parameter scripts        |
 | `objects`           | Generic objects                  |
 | `tasks`             | Queued device management tasks   |
 | `faults`            | Error records with retry state   |
 | `operations`        | In-flight async operations       |
 | `files` (GridFS)    | Firmware images and config files |
 | `permissions`       | RBAC rules                       |
 | `users`             | User accounts                    |
 | `config`            | Key-value configuration          |
 | `cache`             | Distributed cache (TTL index)    |
 | `locks`             | Distributed locks (TTL index)    |
 `db/synth.ts` is the sophisticated expression-to-MongoDB query compiler. It
 normalizes expressions, converts them to a Boolean satisfiability representation
 using the `Clause` hierarchy, minimizes via Espresso, and emits MongoDB
 `$and`/`$or`/`$not` filter objects. This is used by the UI backend.
 `cwmp/db.ts` handles CWMP-specific persistence: loading the nested device
 document into `DeviceData` (`fetchDevice`), diffing changes back into MongoDB
 update operations (`saveDevice`), and managing faults, tasks, and operations.
 ### Query Systems
 There are two separate query paths:
 1. **NBI queries** (`lib/query.ts`) -- Processes MongoDB-style JSON queries from
   external API consumers. Expands parameter names, generates multi-type value
   interpretations, and passes through to MongoDB directly.
 2. **UI/Expression queries** (`lib/db/synth.ts`) -- Compiles the internal
   expression language into optimized MongoDB filters via Boolean minimization.
   This is the more sophisticated path, used by the UI backend.
 ### Frontend (`ui/`)
 A Mithril.js SPA with hash-based routing (`#!/`). Key modules:
 - `app.ts` -- Route definitions. The `pagify()` function wraps each page into a
  `RouteResolver` that handles initialization, error boundaries, and data
  fulfillment.
 - `layout.ts` -- Top-level layout (header, navigation, content, overlay).
 - `store.ts` -- Centralized data store. Implements a query-based reactive cache
  with deduplication and incremental fetching. The `fulfill()` method (called
  after every render) batches pending queries, computes filter diffs via
  `unionDiff()`, and fetches only missing data. Connection monitoring polls
  every 3 seconds for server health, clock skew, and config changes.
 - `components.ts` -- Component registry with a context propagation system. The
  proxied `m()` function resolves string component names and the `m.context()`
  API passes data (like the current device object) down the component tree
  without explicit prop threading.
 - `smart-query.ts` -- Translates user-friendly `Label: value` searches into
  filter expressions with type-aware matching (string, number, timestamp, MAC
  address, tag).
 - `task-queue.ts` -- Two-stage task pipeline: staging (user configures tasks)
  then queue (tasks are committed and executed via the backend).
 - `dynamic-loader.ts` -- Lazy loading of heavy libraries (CodeMirror, YAML) via
  dynamic `import()`.
 ### Build System (`build/`)
 The build is a self-bootstrapping esbuild pipeline (`npm run build` pipes
 `build/build.ts` through esbuild then node). It produces:
 - **Backend binaries** -- 5 entry points bundled for Node.js 12+ with shebang
  banners, executable permissions, and `.js` extension stripped.
 - **Frontend bundle** -- `ui/app.ts` bundled for browsers (ESM, code-split) with
  content-hashed filenames.
 - **CSS** -- `ui/css/app.css` bundled and minified with content-hashed output.
 - **SVG sprite** -- All icons in `ui/icons/` optimized via SVGO and combined
  into a single sprite.
 `build/assets.ts` is a compile-time bridge: at rest it contains placeholder
 filenames; during build, the `assetsPlugin` replaces them with actual
 content-hashed names so both backend and frontend reference the correct assets.
 Build metadata includes the date and a hash derived from the git state, appended
 to the package version.
 ## Cross-Cutting Concerns
 ### Multi-Level Caching
 ```
 memoize.ts         In-process function cache (2-4 min, two-generation rotation)
     |
 local-cache.ts     In-process snapshot cache (5s refresh, hash-based revisions)
     |
 cache.ts           MongoDB-backed distributed cache (configurable TTL)
     |
 MongoDB            TTL index auto-expiration
 ```
 Each service has its own `local-cache` that periodically checks the distributed
 cache for staleness. Distributed locks (`lock.ts`) coordinate rebuilds so only
 one worker does the expensive computation.
 ### Configuration (`lib/config.ts`)
 Three-tier priority: CLI args > environment variables (`GENIEACS_*`) > config
 file (`config.json`) > defaults. Supports per-device overrides by appending
 `-OUI-ProductClass-SerialNumber` to option names.
 ### Distributed Locking (`lib/lock.ts`)
 MongoDB-based mutual exclusion using upsert + duplicate key detection. TTL
 indexes prevent deadlocks from crashed processes. Clock skew tolerance of 30
 seconds.
 ### Authentication
 - **CWMP devices** -- HTTP Basic or Digest auth, configurable per-device via
  expressions (`auth.ts`).
 - **UI users** -- JWT tokens in cookies. Passwords hashed with PBKDF2-SHA512
  (10000 iterations). Role-based authorization via `Authorizer` with
  expression-based filters and validators.
 ### Connection Requests (`lib/connection-request.ts`)
 Three methods to ask a CPE device to initiate a CWMP session:
 - **HTTP** -- Standard TR-069 connection request with Digest/Basic auth.
 - **UDP** -- For NAT traversal (STUN-based) with HMAC-SHA1 signed messages.
 - **XMPP** -- TR-069 Annex K via a full XMPP client (`xmpp-client.ts`).
 ### Extensions (`lib/extensions.ts`)
 User-defined scripts executed in long-lived child processes (`genieacs-ext.ts`)
 communicating via IPC. Processes are lazily spawned per script name and reused.
 Each request gets a unique ID; responses are matched by ID with a configurable
 timeout.
 ### Logging (`lib/logger.ts`)
 Dual-stream structured logging (application + access logs). Supports simple and
 JSON formats, systemd journal integration, and automatic log rotation detection.
 Protocol traces can be written to a debug file in YAML or JSON format
 (`debug.ts`).
 ### Three-Valued Logic
 The system consistently implements SQL-style three-valued logic
 (true/false/null). This is visible in expression evaluation, the `Clause`
 hierarchy's separate true/false/null methods, and the 2-bit minterm encoding in
 the Boolean minimizer. NULL means "unknown" and propagates through operations
 following SQL semantics.
 ## Architectural Invariants
 - **The expression system does not depend on any service-specific code.** The
  `lib/common/expression/` modules are pure and shared across all services.
 - **The sandbox is deterministic across replays.** `Math.random()` is seeded
  from the device ID, `Date.now()` is controlled, and extension results are
  cached. A script re-run with the same inputs produces the same outputs.
 - **Services share no in-process state.** All cross-process coordination goes
  through MongoDB (the `cache` and `locks` collections). Each worker process is
  independent.
 - **Device data is never mutated in place during a session.** `VersionedMap`
  provides revision-based snapshots. The sandbox writes to new revisions;
  earlier revisions remain readable for re-evaluation.
 - **CWMP session exclusivity.** A distributed lock (`cwmp_session_<deviceId>`)
  ensures only one session exists per device at a time. The lock is refreshed
  periodically and released at session end.
 - **The UI backend never queries MongoDB with raw user input.** All queries go
  through the expression-to-MongoDB compiler (`db/synth.ts`), which validates
  and normalizes expressions before generating filters.
@@ -1,378 +0,0 @@
 # Change Log
 ## 1.2.14 (2026-03-12)
 - Prevent UI crash when a malformed URL is sent to the server.
 - Fix potential edge-case bugs in expression evaluation and session
  serialization.
 ## 1.2.13 (2024-06-06)
 - Increase connection timeout for UI and NBI from 30 to 120 seconds to avoid
  timeouts when running unindexed queries in large deployments.
 - Fix race condition causing 503 error when deleting multiple faults at once.
 - Fix some UI config options not being evaluated as dynamic expressions.
 - Fix an issue where certain edge-case query expressions were not being
  correctly converted to MongoDB queries, resulting in inaccurate search
  results.
 ## 1.2.12 (2024-03-28)
 - Fix broken XMPP support in the previous release.
 - Fix regression causing CSV downloads to be buffered in memory before being
  streamed to the client.
 ## 1.2.11 (2024-03-21)
 - Resolved an issue from the previous release that caused incompatibility with
  Node.js versions 12 through 15.
 ## 1.2.10 (2024-03-18)
 - Add support for XMPP connection requests. Use the environment variables
  `XMPP_JID` and `XMPP_PASSWORD` to configure the XMPP connection for the ACS.
 - The environment variables `CWMP_SSL_CERT` and `CWMP_SSL_KEY`, as well as their
  counterparts for UI, NBI, and FS, now accept PEM-encoded certificates and keys
  in addition to file paths.
 - The UI no longer requires users to refresh the page after modifying presets,
  provisions, or virtual parameters. Refreshing is now only necessary for
  changes to users, permissions, or UI configurations.
 - Improved conversion of GenieACS expressions into MongoDB queries for more
  optimized queries and better index utilization.
 - Refactor UI pagination and sorting to fix issues from the previous approach,
  especially with sorting by rapidly changing parameters such as 'last inform'
  time.
 - The file server now supports HEAD requests, the Range header, and conditional
  requests.
 - Addressed an issue causing file server disconnections for slow clients over
  HTTPS.
 - Fix bug preventing users from changing their own passwords.
 - Introduced a new 'locks' collection for database locking, replacing the
  previous use of the 'cache' collection for this purpose.
 - Various other minor fixes and enhancements.
 ## 1.2.9 (2022-08-22)
 - New config option `cwmp.skipRootGpn` to enable a workaround for some
  problematic CPEs that reject GPN requests on data model root.
 - Stream query results and CSV downloads as data becomes available instead of
  buffering the entire response.
 - Log HTTP/HTTPS client errors in debug log.
 - Fix occasional lock expired errors after updating presets, etc.
 - Fix bug where queries containing `<>` operator may return incorrect results.
 - Fix a performance hit caused by DB calls containing the entire CPE data model
  rather then just the updated parameters.
 - Fix bug where tags containing special characters are saved in their encoded
  form when set via a Provision script.
 - Fix issue causing invalid `xsd:dateTime` values to be saved in DB as `NaN`
  rather than maintain the original string value.
 - Fix bug using `$regex` operator with a numeric or a datetime string.
 - Fix ping not working on certain platforms.
 - Ping requests are now authenticated.
 - Fix error when the `FORWARDED_HEADER` environment variable contains IPv4 CIDR
  while the listening interface is IPv6 and vice versa.
 - Fix false warning "unexpected parameter in response" showing in
  `genieacs-cwmp` logs.
 - Other minor fixes and stability improvements.
 ## 1.2.8 (2021-10-27)
 - Fix a remote code execution security vulnerability in genieacs-ui.
 - All UI components can now be configured using fully dynamic expressions.
  Previously some component types only accept fixed string values as properties.
 - The `container` UI component can now be configured with an `element` property
  that's either a string or a nested pair of `tag` and `attributes` properties.
  The various attributes under the `attributes` property can now make use of a
  new function `ENCODEURICOMPONENT()` to facilitate creating custom hyperlinks
  in the UI.
 - Improve sorting buttons' behavior in the device listing page and other pages.
  Sorting by multiple columns should now feel more intuitive.
 - Support the modulo (%) operator in expressions.
 - Fix a regression in the previous release where the config option 'ui.pageSize'
  no longer works.
 - Fix process crash when a CPE sends an unsupported ACS method.
 ## 1.2.7 (2021-09-18)
 - Fix regression causing frequent invalid session errors.
 - Fix regression breaking digest authentication in connection requests.
 ## 1.2.6 (2021-09-16)
 - New config option `cwmp.downloadSuccessOnTimeout` to enable a workaround for
  CPEs that neglect to send a TransferComplete message after a successful
  download.
 - Display a progress bar when uploading new files.
 - Default to dual stack interface binding (i.e. `::` instead of `0.0.0.0`).
  Unless the binding interface is explicitly set, this will cause IPv4 addresses
  in the logs to be displayed as IPv4-mapped IPv6 addresses (e.g.
  `::ffff:127.0.0.1` instead of `127.0.0.1`).
 - Detect and correct for client side (browser) clock skew that would otherwise
  alter the numbers in the pie charts.
 - Various improvements relating to dealing with buggy TR-069 client
  implementations.
 - Fix a bug causing "lock expired" exceptions when a CWMP session remains open
  for a very long time due to slow clients.
 - Fix metadata of uploaded files going missing due to nginx stripping away what
  it considers to be invalid headers. The nginx directive
  `ignore_invalid_headers` is no longer required.
 - Fix crash when a CPE is assigned a tag containing a dot.
 - Fix bug preventing the user from closing the preset pop-up after being
  presented with unsupported preset message.
 - Fix exceptions raised from ext scripts manifesting as timeout faults.
 - Fix download getting triggered repeatedly when the value passed to `Download`
  parameter is greater than the current timestamp.
 - Fix crash when passing invalid attributes to `declare`.
 - Fix crash in NBI when pushing tasks to non-existing devices.
 - Fix crash in NBI when passing invalid JSON to various API endpoints.
 - Fix crash when the output from `ping` command cannot be parsed in some rare
  edge cases.
 - A number of other fixes and stability improvements.
 ## 1.2.5 (2021-03-12)
 - Support specifying custom types when uploading files.
 - Fix JS compatibility issue with Safari browser.
 ## 1.2.4 (2021-02-24)
 - The data model state of a CPE is no longer forgotten after unsuccessful
  session termination (e.g. timeout). This addresses a number of undesired side
  effects that arise when a CPE does not terminate the session properly.
 - Executing tasks that take a long time to complete (e.g. refreshing the entire
  data model) no longer shows a timeout error while the task is still being
  processed.
 - New function `ROUND()` available to expressions. It works similar to the
  function by the same name in SQLite and PostgreSQL.
 - Log access events for `genieacs-nbi` service.
 - Pipe stdout/stderr from extension scripts to the `genieacs-cwmp` process log.
 - Parameter values of type `xsd:dateTime` are now displayed in the UI and CSV
  downloads as a date string rather than a numeric value.
 - Add file download link in the files listing page.
 - Display spinner loading animation throughout the UI.
 - Display GenieACS version and build number underneath the logo.
 - New option to specify how many parameters are displayed at a time in the
  all-parameters component. Simply set `limit` property in the component config.
 - Reduce overly strong Brotli compression level which was causing significant
  page load slowdown when Brotli is used.
 - Retire dump-data-model tool. `genieacs-sim` can now use a CSV file as its data
  model.
 - Reduce the number of concurrent database connections from each process.
 - Remove dependency on 'hostInfo' MongoDB command which is a privileged action.
  It is now possible to use shared MongoDB instances with limited privileged.
 - Fix bug in NBI where querying files returns 404 error.
 - Fix ping not working for devices with an IPv6 address.
 - Fix an elusive memory leak in `genieacs-fs` that slowly eats up memory and can
  go unnoticed for long periods of time.
 - Fix a rare edge case where a `declare()` call to set a parameter value may not
  work as intended if the parameter was originally received as part of the
  Inform message.
 - A number of other fixes and stability improvements.
 ## 1.2.3 (2020-10-26)
 - New config option 'cwmp.skipWritableCheck' for when some CPEs incorrectly
  report writable parameters as non-writable. When set to true, the scripts will
  no longer respect the 'writable' attribute of the CPE parameters and will send
  a SetParamteerValues, AddObject, or DeleteObject request anyway.
 - Tags no longer restrict what characters are allowed. Any character other than
  alphanumeric characters, hyphen, or underscore is now encoded in the data
  model (i.e. Tags.\<tag>) using its hex value preceded by "0x".
 - Ask for a confirmation before closing a pop-up dialog with unsaved changes.
 - Better XML validation to avoid crashes caused by invalid CPE requests.
 - Fix confusing 404 error message when the user attempts to modify a resource
  when they don't have the necessary permissions.
 - Fix a rare issue where genieacs-cwmp stops accepting new connections after
  running for a few weeks.
 - Fix exception when IS NULL operator is used in certain situations.
 ## 1.2.2 (2020-10-03)
 - Added button to push files to selected devices from device listing page.
 - A few minor UI improvements.
 - Fix exception that can happen and persist after a Download request.
 - Fix validation bug preventing running refreshObject task on data model root.
 - Fix invalid arguments fault in refresh preset configuration when upgrading
  from v1.1.
 ## 1.2.1 (2020-09-08)
 - Fix bug causing faults to not be displayed in the UI.
 - Fix bug where deleting objects does not get reflected immediately in the UI.
 - Improve conversion between filters written in the expression format and
  MongoDB queries. There should now be fewer edge cases where the two are not
  equisatisfiable.
 ## 1.2.0 (2020-09-01)
 - Support GetParameterAttributes and SetParameterAttributes TR-069 methods.
 - Support CASE statement and COALESCE function in expressions.
 - Provision arguments can now be a list of expressions that are dynamically
  evaluated.
 - Support Forwarded HTTP header to display in the logs the correct IP of CPEs
  behind a reverse proxy. Must be configured using FORWARDED_HEADER option.
 - Config expressions can now access all available device parameters, not only
  serial number, product class, and OUI.
 - Use relative URLs throughout the UI to allow serving from a subdirectory using
  a reverse proxy.
 - Make Date.parse() and Date.UTC() available to provision scripts.
 - libxmljs has been entirely removed in favor of our bespoke XML parser.
 - Removed the config option CWMP_KEEP_ALIVE_TIMEOUT. SESSION_TIMEOUT is now used
  to determine the TCP connection timeout.
 - The all-parameters component now limits the number of parameters displayed for
  better performance.
 - The process genieacs-cwmp is now much less likely to throw exceptions as a
  result of invalid requests from CPE.
 - A large number of bug fixes and stability improvements.
 ## 1.2.0-beta.0 (2019-07-30)
 - A brand new UI superseding genieacs-gui.
 - New initialization wizard on first run.
 - New expression/query language used in search filters and preset preconditions.
 - CPE -> ACS authentication is now supported.
 - New config option (CWMP_KEEP_ALIVE_TIMEOUT) to specify how long to wait for a
  reply from the CPE before closing the TCP connection.
 - Debug logging has been reimplemented utilizing YAML format for logs.
 - Handle 9005 faults (Invalid Parameter Name) gracefully by attempting to
  rediscover the path of the missing parameter recursively.
 - declare() statements not followed by an explicit commit() are now deferred
  until all currently active scripts have been executed.
 - FS_HOSTNAME now defaults to the server's hostname or IP.
 - The API now validates the structure of task objects before saving.
 - New XML parser implementation for better performance. You can revert to the
  old parser by enabling the config option XML_LIBXMLJS. Requires Node.js v11 or
  v10.
 - Performance optimizations. While performance has improved for the majority of
  use cases, there may be situations where performance has degraded. It's
  recommended to revisit your hardware requirements.
 - Connection request authentication no longer uses 'auth.js' file. Instead, the
  connection request authentication behavior can now be customized using an
  'expression'.
 - The config file (config.json) has been deprecated. System configuration (e.g.
  listen ports, worker count) are now recommended to be passed as environment.
  variables. Other general configuration options are stored in the database so
  as to not require service restart for changes to take effect.
 - Optional redis dependency has been removed completely.
 - Tags now allow only alphanumeric characters and underscore.
 - Supported versions of Node.js and MongoDB are 10.x and up and 2.6 and up
  respectively.
 ## 1.1.3 (2018-10-23)
 - New config option (MAX_COMMIT_ITERATIONS) to avoid max commit iterations
  faults for more complex scripts.
 - Support base64 and hexBinary parameter types.
 - Strict parsing of number values in queries (e.g. "123abc" no longer accepted
  as 123).
 - Mixing $ne and $not operators is not allowed. Now it throws an error instead
  of returning incorrect results.
 - When a task expires, any associated fault is also deleted.
 - API now accepts 'timeout' argument when posting a task.
 - A number of stability fixes.
 ## 1.1.2 (2018-02-24)
 - A large number of bug fixes as well as stability and performance improvements.
 - Three security vulnerabilities disclosed by Maximilian Hils have been patched.
 - New config option UDP_CONNECTION_REQUEST_PORT to specify binding port for UDP
  connection requests.
 - New config option DATETIME_MILLISECONDS to strip milliseconds from dateTime
  values.
 - New config option BOOLEAN_LITERAL to use 1/0 or true/false for boolean values.
 - Parameter values that cannot be parsed according to the reported type now show
  a warning message.
 - Virtual parameter scripts now use the variable 'args' instead of the special
  'TIMESTAMPS' and 'VALUES' variables. The content of the args array is:
  {declare timestamps}, {declare values}, {current timestamps}, {current
  values}.
 - Virtual parameter value types are now inferred from the JavaScript type if the
  returned value attribute is not a value-type pair.
 - Show a fault when a virtual parameter script doesn't return the required
  attributes.
 - Redis is now optional (and disabled by default), reducing the complexity of
  scalable deployments.
 - Better detection of cyclical presets resulting in fewer faults for complex
  provisioning scripts.
 - Math.random() is now deterministic on per-device basis. A function has been
  added to allow specifying a seed value (e.g. Math.random.seed(Date.now())).
 - Overload spikes are now handled gracefully by refusing to accept new sessions
  temporarily when under abnormal load.
 - Added log messages for session timeouts, connection drops, and XML parsing
  errors.
 - Date.now() now takes an optional argument to specify "time steps" (in
  milliseconds). This can be used to ensure a group of parameters are all
  refreshed at the same time intervals.
 - Only the non-default configuration options are now logged at process start.
 - Faults caused by errors from extensions now show a cleaner stack trace.
 - Exit main process if there are too many worker crashes (e.g. when DB is down).
 - Updated dependencies and included a lockfile to ensure installations get the
  exact dependencies it was tested against.
 ## 1.1.1 (2017-03-23)
 - Avoid crashing when connection request credentials are missing.
 - Show a warning instead of crashing when failing to parse parameter values
  according to the expected value type.
 - Add missing "Registered" event.
 - Fix bug where in certain cases many more instances than declared are created.
 - Fix parameter discovery bug when declared path timestamp is 1 or is not set.
 - Fix preset precondition failing when testing against datetime parameters and
  certain other parameters like \_deviceId.\_ProductClass.
 ## 1.1.0 (2017-03-10)
 - Provisions enable implementing dynamic device configuration or complex device
  provisioning work flow using arbitrary scripts.
 - Virtual parameters are user-defined parameters whose values are evaluated from
  a custom script.
 - Extensions are sandboxed Node.js scripts that are accessible from provision
  and virtual parameter scripts to facilitate integration with external
  entities.
 - Support for UDP/STUN based connection requests for reaching devices behind NAT
  (TR-069 Annex G).
 - Presets can now be scheduled using a cron-like expression.
 - Presets can now be tied to specific device events (e.g. boot).
 - Presets precondition queries no longer support "\$or" or other MongoDB logical
  operators.
 - Faults are no longer a part of tasks but are now first class objects.
 - Presets are now assigned to channels. A fault in one channel only blocks
  presets in that channel.
 - New API CRUD functions for provisions, virtual parameters, and faults.
 - New config options for XML output.
 - API responses now include "GenieACS-Version" header.
 - Graceful shutdown when receiving SIGINT and SIGTERM events.
 - Support SSL intermediate certificate chains.
 - Supported Node.js versions are 6.x and 7.x.
 - Supported MongoDB versions are 2.6 through 3.4.
 - Expect performance differences due to major under the hood changes. Some
  operations are faster and some are slower. Overall performance is improved.
 - GenieACS will no longer fetch the entire device data model upon first contact
  but will instead only fetch the parameters it needs to fulfill the presets.
 - Logs have been overhauled and split into two streams: process log (stderr) and
  access log (stdout). Also added config options to dump logs to files rather
  than standard streams.
 - Connection request authentication credentials are picked up from the device
  data model if available. config/auth.js is still supported as a fallback and
  now supports an optional callback argument.
 - Custom commands have been removed. Use virtual parameters and/or extensions.
 - Aliases and value normalizers (config/parameters.json) have been removed. Use
  virtual parameters.
 - The API /devices/<device_id>/preset has been removed.
 - Rarely used RequestDownload method no longer supported.
 - The TR-069 client simulator has moved to its own repo at
  https://github.com/zaidka/genieacs-sim
@@ -1,352 +0,0 @@
 # Contributing to GenieACS
 ## Questions and Support
 Please use the [forum](https://forum.genieacs.com) for questions, help requests,
 and general discussion. GitHub Issues are reserved for confirmed bug reports.
 ## Issues
 We use GitHub Issues to track bugs. When filing a bug report:
 - Provide a clear description of the problem.
 - Include steps to reproduce the issue.
 - Note the GenieACS version, Node.js version, and MongoDB version.
 - Include relevant log output if applicable.
 If you face interoperability issues with a CPE, it is more often than not a
 device-specific issue. Please consult the [forum](https://forum.genieacs.com)
 before opening an Issue.
 ## Pull Requests
 Pull requests are welcome. For bug fixes, go ahead and open a PR directly. For
 new features or significant changes, please discuss in the
 [forum](https://forum.genieacs.com) first to ensure alignment with the project's
 direction.
 When submitting a PR:
 - Keep changes focused. One PR should address one concern.
 - Run `npm run lint` and `npm test` before submitting.
 - Update documentation in `docs/` if your change affects user-facing behavior.
 - Add tests where appropriate (see [Testing](#testing)).
 - Write a clear PR description explaining what the change does and why.
 ## Code Style
 ### Naming Conventions
 - **Variables and functions**: `camelCase`
 - **Classes**: `PascalCase`
 - **Interfaces and type aliases**: `PascalCase`, without an `I` or `T` prefix
 - **Module-level constants**: `UPPER_SNAKE_CASE`
 - **Private class members**: prefixed with underscore (`_name`, `_cache`)
 Choose descriptive, meaningful names. A longer clear name is better than a short
 ambiguous one.
 ### Imports
 Organize imports in three groups, in this order:
 1. Node.js built-in modules, using the `node:` prefix
 2. External packages
 3. Local project modules
 Use `.ts` extensions for all local imports.
 ```typescript
 import { readFileSync } from "node:fs";
 import * as http from "node:http";
 import { Collection } from "mongodb";
 import { connect, disconnect } from "./db.ts";
 import Path from "./common/path.ts";
 ```
 ### Functions and Exports
 Use the `function` keyword for all named and exported function declarations.
 Arrow functions should only be used for callbacks and short inline expressions.
 ```typescript
 // Named/exported functions use function keyword
 export function processRequest(req: Request): Response {
  // ...
 }
 // Arrow functions for callbacks
 items.filter((item) => item.active);
 promise.then((result) => {
  // ...
 });
 ```
 ### TypeScript
 - All function declarations must have explicit return types. This is enforced by
  ESLint.
 - Using `any` is permitted where full typing would be impractical, but prefer
  more specific types when reasonable.
 - Prefer `Record<string, T>` over `{ [key: string]: T }` for mapped types.
 - Use type assertions (`as`) sparingly and only when you have stronger type
  knowledge than the compiler.
 ### Comments
 Code should be self-documenting. Use comments sparingly and only when the "why"
 is not obvious from the code itself. When you do comment, use inline `//` style.
 Do not use JSDoc (`/** */`) or block comments (`/* */`).
 ```typescript
 // Good: explains a non-obvious reason
 // Escapes everything except alphanumerics and underscore
 function escapeRegExp(str: string): string {
  /* ... */
 }
 // Good: links to an external reference
 // Source: http://stackoverflow.com/a/6969486
 // Good: flags a known limitation
 // TODO support "MD5-sess" algorithm directive
 // Bad: restates what the code does
 // Loop through each item and increment the counter
 for (const item of items) {
  counter++;
 }
 ```
 ### Error Handling
 Use `== null` (not `=== null || === undefined`) for null/undefined checks.
 ESLint is configured to allow this.
 ## Testing
 ### When to Write Tests
 Tests are most valuable for pure logic: parsers, data transformations,
 algorithms, and utility functions where inputs and outputs are well defined. A
 regression test accompanying a bug fix is also worthwhile to prevent the same
 issue from resurfacing.
 Not every change needs a test. Use judgment and consider the likelihood and cost
 of breakage. Avoid writing tests that duplicate coverage already provided by
 existing tests, and resist the urge to test trivial code just for the sake of
 coverage numbers.
 ### Conventions
 Tests use the Node.js built-in test runner (`node:test`) and assertion module
 (`node:assert`). No external test libraries or mocking frameworks are used.
 ```typescript
 import test from "node:test";
 import assert from "node:assert";
 void test("parseValue returns correct type for integer strings", () => {
  const result = parseValue("42");
  assert.strictEqual(result, 42);
 });
 void test("parseValue throws on invalid input", () => {
  assert.throws(() => parseValue(""), new Error("empty value"));
 });
 ```
 Key conventions:
 - Prefix `test()` calls with `void` to satisfy the `no-floating-promises` lint
  rule.
 - Keep tests flat. Do not nest `describe` blocks.
 - Use descriptive test names that state what is being tested and the expected
  outcome.
 - Use `assert.strictEqual()` for value comparisons and
  `assert.deepStrictEqual()` for objects and arrays.
 Test files live in the `test/` directory and are named after the module they
 test (e.g. `test/path.ts` tests `lib/common/path.ts`).
 ## Dependencies
 This project deliberately keeps its dependency footprint small. Before adding a
 new dependency:
 - Prefer Node.js built-in APIs when they can do the job.
 - Consider whether the functionality is simple enough to implement directly.
 - Justify the addition in your PR description.
 Do not add development tool dependencies (linter plugins, editor integrations,
 etc.) without prior discussion.
 ## File Organization
 | Directory     | Contents                                     |
 | ------------- | -------------------------------------------- |
 | `lib/`        | Core server-side application code            |
 | `lib/common/` | Code shared between server and browser       |
 | `lib/db/`     | Database layer (MongoDB)                     |
 | `lib/ui/`     | UI backend helpers                           |
 | `ui/`         | Frontend code (Mithril.js SPA)               |
 | `bin/`        | Service entry points                         |
 | `build/`      | Build tooling                                |
 | `test/`       | Test files                                   |
 | `docs/`       | User documentation (Sphinx/reStructuredText) |
 | `public/`     | Static assets (favicon, logo)                |
 Place new code in the directory that matches its purpose. Server-side logic
 belongs in `lib/`, code that must run in both Node.js and the browser belongs in
 `lib/common/`, and frontend-only code belongs in `ui/`.
 ## Documentation
 User documentation lives in the `docs/` directory as reStructuredText files
 built with Sphinx and published to
 [docs.genieacs.com](https://docs.genieacs.com).
 When your change affects user-facing behavior:
 - Update the relevant documentation in `docs/`.
 - If adding a new feature, consider whether it warrants a new page or a section
  in an existing page.
 - Keep documentation concise and practical. Match the existing tone: direct,
  factual, no filler.
 Documentation changes should be included in the same PR as the code change, not
 submitted separately.
 ### ARCHITECTURE.md
 `ARCHITECTURE.md` describes the high-level architecture of the project: the
 service boundaries, major subsystems, data flow, and key invariants. It is aimed
 at contributors who need a mental map of the codebase.
 This file has a different update cadence than `docs/`. It should be revisited a
 few times a year rather than kept in lockstep with every code change. When you
 do update it, follow these principles:
 - **Only describe things that are unlikely to change frequently.** Module
  responsibilities, service boundaries, key data structures, and architectural
  invariants belong here. Implementation details, function signatures, and
  config option lists do not.
 - **Name important files, modules, and types but do not link them.** Links go
  stale. Encourage the reader to use symbol search to find named entities; this
  also helps them discover related, similarly named things.
 - **Keep it short.** Every recurring contributor will read it. A shorter
  document is less likely to become stale and more likely to be maintained.
 - **Describe the "what" and "where", not the "how".** This is a map of the
  country, not an atlas of its states. Pull detailed explanations into inline
  code comments or separate documents.
 - **Call out architectural invariants explicitly.** Important invariants are
  often expressed as the _absence_ of something (e.g., "the expression system
  does not depend on service-specific code") and are hard to discover by reading
  code alone.
 ## Git Workflow
 ### Commit Messages
 This project follows the
 [Conventional Commits](https://www.conventionalcommits.org/) format:
    <type>(<scope>): <subject>
    [optional body]
 - Use the imperative mood, present tense: "Fix bug", not "Fixed bug" or "Fixes
  bug".
 - Do not capitalize the first letter of the subject (the type prefix handles
  visual structure).
 - Do not end the subject line with a period.
 - Keep the subject line under 72 characters.
 - When more context is helpful, add a body separated from the subject by a blank
  line, wrapped at 72 characters. The goal is to provide enough information for
  someone scanning the commit history to find a specific change (e.g. for
  troubleshooting or rebasing) or to draft changelog entries for a release.
  Don't be verbose, but don't be cryptic either.
 #### Types
 | Type       | When to use                                                                 |
 | ---------- | --------------------------------------------------------------------------- |
 | `fix`      | Bug fixes                                                                   |
 | `feat`     | New features or capabilities                                                |
 | `refactor` | Code restructuring with no behavior change                                  |
 | `test`     | Adding or updating tests                                                    |
 | `docs`     | Documentation changes (`docs/`, `CONTRIBUTING.md`, `README.md`)             |
 | `build`    | Build system, dependencies, esbuild config, `package.json` scripts          |
 | `chore`    | Maintenance tasks that don't fit above (`.gitignore`, tooling config, etc.) |
 #### Scopes
 Scope is optional. Use it when it adds useful context; omit it when the change
 is cross-cutting or the subject already makes it obvious.
 | Scope  | Covers                              |
 | ------ | ----------------------------------- |
 | `cwmp` | CWMP service (including extensions) |
 | `nbi`  | Northbound REST API service         |
 | `fs`   | File service                        |
 | `ui`   | Web UI (frontend and backend)       |
 | `db`   | Database layer                      |
 If a change touches multiple scopes, either pick the primary one or omit the
 scope entirely.
 #### Examples
    feat(nbi): add bulk device delete endpoint
    fix(cwmp): handle missing ParameterKey in InformResponse
    refactor(db): replace raw queries with parameterized calls
    fix(ui): correct parameter table sort order
    test: add XML parser edge case coverage
    docs: update provisioning guide for new API
    build: upgrade esbuild to v0.20
    fix(cwmp): increase server timeout to 2 mins
    To allow enough time for running unindexed queries in large
    deployments.
 ### Branches
 - `master` is the main development branch.
 - Create a feature or fix branch for your work and open a PR against `master`.
 - Use concise, descriptive branch names in lowercase with hyphens:
  `fix-race-condition`, `support-xmpp-requests`.
 ## Changelog
 The changelog (`CHANGELOG.md`) is maintained for each release and is written for
 users and system administrators, not developers.
 - Write entries as clear, user-facing prose. Do not copy commit messages
  verbatim.
 - Each entry should describe what changed and, when helpful, why it matters.
 - Group entries under a version heading with the release date:
  `## 1.2.13 (2024-06-06)`.
 - Start each entry with a verb: "Fix", "Add", "Improve", "Remove", etc.
 - Include enough context that a user can understand the impact without reading
  the code.
 You do not need to update the changelog in your PR. The maintainer will add
 changelog entries during the release process.
 ## Contributor License Agreement
 By submitting a pull request to this repository, you acknowledge that, while
 maintaining copyright, you grant GenieACS Inc. a perpetual, worldwide,
 non-exclusive, no-charge, royalty-free, irrevocable license to reproduce,
 prepare derivative works of, publicly display, publicly perform, sublicense, and
 distribute your contributions and such derivative works under the AGPLv3 license
 or any other license terms, including, but not limited to, proprietary or
 commercial license terms.
 You confirm that you own or have rights to distribute and sublicense the source
 code contained therein, and that your content does not infringe upon the
 intellectual property rights of a third party.
@@ -1,68 +0,0 @@
 # =========================================================
 # GenieACS — UniFi Theme Fork
 # Multi-stage build:
 #   Stage 1 (builder): instala devDeps + compila o projeto
 #   Stage 2 (runtime): imagem mínima só com dist/ + prodDeps
 # =========================================================
 # ── Stage 1: builder ─────────────────────────────────────
 FROM node:20-alpine AS builder
 # Dependências de sistema necessárias para o build
 # (git é exigido pelo build.ts para calcular o metadata de versão)
 RUN apk add --no-cache git python3 make g++
 WORKDIR /build
 # Copiar manifests primeiro para cache de camadas do npm install
 COPY package.json npm-shrinkwrap.json ./
 # Instalar TODAS as dependências (incluindo devDependencies)
 RUN npm ci --fund=false --audit=false
 # Copiar o restante do fonte
 COPY . .
 # Compilar o projeto (gera dist/)
 RUN npm run build
 # ── Stage 2: runtime ─────────────────────────────────────
 FROM node:20-alpine AS runtime
 # Timezone e utilitários mínimos
 RUN apk add --no-cache tzdata dumb-init
 WORKDIR /opt/genieacs
 # Copiar artefatos compilados
 COPY --from=builder /build/dist ./
 # Instalar apenas dependências de produção a partir do dist/
 RUN npm ci --omit=dev --fund=false --audit=false
 # Criar diretórios de configuração e extensões
 RUN mkdir -p \
      /opt/genieacs/config/ext \
      /opt/genieacs/logs \
    && addgroup -S genieacs \
    && adduser  -S -G genieacs genieacs \
    && chown -R genieacs:genieacs /opt/genieacs
 USER genieacs
 # Variáveis de ambiente padrão (sobrescreva no compose ou .env)
 ENV GENIEACS_MONGODB_CONNECTION_URL=mongodb://mongo:27017/genieacs \
    GENIEACS_EXT_DIR=/opt/genieacs/config/ext \
    GENIEACS_LOG_FORMAT=json \
    GENIEACS_ACCESS_LOG_FORMAT=json \
    GENIEACS_UI_JWT_SECRET=change-me-in-production \
    GENIEACS_CWMP_WORKER_PROCESSES=1 \
    GENIEACS_NBI_WORKER_PROCESSES=1 \
    GENIEACS_FS_WORKER_PROCESSES=1 \
    GENIEACS_UI_WORKER_PROCESSES=1
 # Expor todas as portas dos serviços
 EXPOSE 3000 7547 7557 7567
 # dumb-init garante propagação correta de sinais
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
@@ -1,80 +1,65 @@
 # GenieACS
-**This is the development branch for GenieACS v1.3. It is unstable and not ready
+GenieACS is a *blazing fast* TR-069 auto configuration server (ACS) built with Node.js, Redis, and MongoDB. It's technology neutral and configurable to fit any service provider needs. This is the core back end component. Official front end GUI is available at https://github.com/zaidka/genieacs-gui.
 for production use. For the latest stable release, see the
 [v1.2 branch](https://github.com/genieacs/genieacs/tree/v1.2).**
-GenieACS is a high performance Auto Configuration Server (ACS) for remote
+## Features
 management of TR-069 enabled devices. It utilizes a declarative and fault
 tolerant configuration engine for automating complex provisioning scenarios at
 scale. It's battle-tested to handle hundreds of thousands and potentially
 millions of concurrent devices.
-## Quick Start
+* **Massively concurrent**: Can handle hundreds of thousands of connected devices on a single server even with low inform interval.
 * **Preset-based configuration**: Define sets of configurations that devices will pick up and apply as needed based on given preconditions.
 * **Tagging**: Use tags to group devices for more manageable presets.
 * **Searching**: Query for devices on any parameter (supports all common operators including regular expressions).
 * **Parameter aliasing**: Define aliases to unify parameter paths from different types of devices. Aliases behave like normal parameters (i.e. can be queried or modified).
 * **Extensive API**: A simple yet rich HTTP-based API allows easy integration with other system.
-Install [Node.js](http://nodejs.org/) and [MongoDB](http://www.mongodb.org/).
+## Getting started
 Refer to their corresponding documentation for installation instructions. The
 supported versions are:
- Node.js: 12.3+
+Install [Node.js](http://nodejs.org/), [Redis](http://redis.io/), and [MongoDB](http://www.mongodb.org/). Refer to the corresponding documentation for installation guides. Then use NPM to install GenieACS by typing:
 - MongoDB: 3.6+
-Install GenieACS from NPM:
+    npm install -g genieacs
-    sudo npm install -g genieacs
+You may need to modify the configuration files under "config" directory (in /lib/node_modules/genieacs/config) depending on your setup.
-To build from source instead, clone this repo or download the source archive
+Alternatively, you can install from source by cloning the git repository:
 then _cd_ into the source directory then run:
    cd /opt
    git clone https://github.com/zaidka/genieacs.git --branch v1.0
    cd genieacs
    npm install
-    npm run build
+    npm run configure
    npm run compile
-Finally, run the following services (found under `./dist/bin/` if building from
+Finally, run the following in GNU Screen session or something similar:
 source):
-### genieacs-cwmp
+    genieacs-cwmp
-This is the service that the CPEs will communicate with. It listens on port 7547
+This is the service that the CPEs will communicate with. It listens to port 7547 by default (see config/config.json). Configure the ACS URL of your devices accordingly.
 by default. Configure the ACS URL in your devices accordingly.
-You may optionally use [genieacs-sim](https://github.com/genieacs/genieacs-sim)
+    genieacs-nbi
 as a dummy TR-069 simulator if you don't have a CPE at hand.
-### genieacs-nbi
+This is the northbound interface module. It exposes a REST API on port 7557 by default. This is needed for the GUI front end to communicate with.
-This is the northbound interface module. It exposes a REST API on port 7557 by
+    genieacs-fs
 default. This one is only required if you have an external system integrating
 with GenieACS using this API.
-### genieacs-fs
+This is the file server from which the CPEs will download firmware images and other types of files.
-This is the file server from which the CPEs will download firmware images and
+You can use stream redirection to output to log files:
 such. It listens on port 7567 by default.
-### genieacs-ui
+    genieacs-cwmp >> /var/log/genieacs-cwmp.log 2>> /var/log/genieacs-cwmp-err.log
    genieacs-nbi >> /var/log/genieacs-nbi.log 2>> /var/log/genieacs-nbi-err.log
    genieacs-fs >> /var/log/genieacs-fs.log 2>> /var/log/genieacs-fs-err.log
-This serves the web based user interface. It listens on port 3000 by default.
+For further details about installation and configuration, refer to the [wiki section](https://github.com/zaidka/genieacs/wiki).
 You must pass _--ui-jwt-secret_ argument to supply the secret key used for
 signing browser cookies:
-    genieacs-ui --ui-jwt-secret secret
+You may now proceed with installing [GenieACS GUI front end](https://github.com/zaidka/genieacs-gui).
 The UI has plenty of configuration options. When you open GenieACS's UI in a
 browser you'll be greeted with a database initialization wizard to help you
 populate some initial configuration.
 Visit [docs.genieacs.com](https://docs.genieacs.com) for more documentation and
 a complete installation guide for production deployments.
 ## Support
-The [forum](https://forum.genieacs.com) is a good place to get guidance and help
+The [Users mailing list](http://lists.genieacs.com) is a good place to get guidance and help from the community. Head on over and join the conversation! In addition, the [wiki](https://github.com/zaidka/genieacs/wiki) provides useful documentation and tips from GenieACS users.
 from the community. Head on over and join the conversation!
-For commercial support options, please visit
+You may submit bug reports or feature requests [here](https://github.com/zaidka/genieacs/issues). For device interoperability issues, please consult the mailing list first — it's likely that a workaround already exists.
 [genieacs.com](https://genieacs.com/support/).
-## License
+For commercial support options and professional services, please visit [genieacs.com](https://genieacs.com).
-Copyright 2013-2026 GenieACS Inc. GenieACS is released under the
+## Contributing
-[AGPLv3 license terms](https://raw.githubusercontent.com/genieacs/genieacs/master/LICENSE).
+
 Contributions are welcome. Fork this repo and open a pull request and wait for feedback. You can also contribute by enhancing the documentation in the [wiki section](https://github.com/zaidka/genieacs/wiki).
@@ -0,0 +1,9 @@
 #!/usr/bin/env node
 var path = require('path');
 var fs = require('fs');
 var dir = path.resolve(path.dirname(fs.realpathSync(__filename)), '..');
 process.chdir(dir);
 cluster = require(path.resolve(dir, 'lib/cluster'));
 cluster.start('cwmp');
@@ -1,118 +0,0 @@
 import * as config from "../lib/config.ts";
 import * as logger from "../lib/logger.ts";
 import * as cluster from "../lib/cluster.ts";
 import * as server from "../lib/server.ts";
 import * as cwmp from "../lib/cwmp.ts";
 import * as db from "../lib/db/db.ts";
 import * as extensions from "../lib/extensions.ts";
 import { version as VERSION } from "../package.json";
 logger.init("cwmp", VERSION);
 const SERVICE_ADDRESS = config.get("CWMP_INTERFACE") as string;
 const SERVICE_PORT = config.get("CWMP_PORT") as number;
 function exitWorkerGracefully(): void {
  setTimeout(exitWorkerUngracefully, 5000).unref();
  Promise.all([
    db.disconnect(),
    extensions.killAll(),
    cluster.worker.disconnect(),
  ])
    .then(logger.close)
    .catch(exitWorkerUngracefully);
 }
 function exitWorkerUngracefully(): void {
  void extensions.killAll().finally(() => {
    process.exit(1);
  });
 }
 if (!cluster.worker) {
  const WORKER_COUNT = config.get("CWMP_WORKER_PROCESSES") as number;
  logger.info({
    message: `genieacs-cwmp starting`,
    pid: process.pid,
    version: VERSION,
  });
  cluster.start(WORKER_COUNT, SERVICE_PORT, SERVICE_ADDRESS);
  process.on("SIGINT", () => {
    logger.info({
      message: "Received signal SIGINT, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
  process.on("SIGTERM", () => {
    logger.info({
      message: "Received signal SIGTERM, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
 } else {
  const key = config.get("CWMP_SSL_KEY") as string;
  const cert = config.get("CWMP_SSL_CERT") as string;
  const options = {
    port: SERVICE_PORT,
    host: SERVICE_ADDRESS,
    ssl: key && cert ? { key, cert } : null,
    onConnection: cwmp.onConnection,
    onClientError: cwmp.onClientError,
    timeout: 30000,
    keepAliveTimeout: 0,
    requestTimeout: cwmp.REQUEST_TIMEOUT,
  };
  // Need this for Node < 15
  process.on("unhandledRejection", (err) => {
    throw err;
  });
  process.on("uncaughtException", (err) => {
    if ((err as NodeJS.ErrnoException).code === "ERR_IPC_DISCONNECTED") return;
    logger.error({
      message: "Uncaught exception",
      exception: err,
      pid: process.pid,
    });
    server.stop(false).then(exitWorkerGracefully).catch(exitWorkerUngracefully);
  });
  const initPromise = db
    .connect()
    .then(() => {
      server.start(options, cwmp.listener);
    })
    .catch((err) => {
      setTimeout(() => {
        throw err;
      });
    });
  process.on("SIGINT", () => {
    void initPromise.finally(() => {
      server
        .stop(false)
        .then(exitWorkerGracefully)
        .catch(exitWorkerUngracefully);
    });
  });
  process.on("SIGTERM", () => {
    void initPromise.finally(() => {
      server
        .stop(false)
        .then(exitWorkerGracefully)
        .catch(exitWorkerUngracefully);
    });
  });
 }
@@ -1,83 +0,0 @@
 import { Fault } from "../lib/types.ts";
 const jobs = new Set();
 const fileName = process.argv[2];
 let script;
 function errorToFault(err: Error): Fault {
  if (!err) return null;
  if (!err.name) return { code: "ext", message: `${err}` };
  const fault: Fault = {
    code: `ext.${err.name}`,
    message: err.message,
    detail: {
      name: err.name,
      message: err.message,
    },
  };
  if (err.stack) {
    fault.detail["stack"] = err.stack;
    // Trim the stack trace
    const stackTrimIndex = fault.detail["stack"].match(
      /\s+at\s[^\s]+\s\(.*genieacs-ext:.+\)/,
    );
    if (stackTrimIndex) {
      fault.detail["stack"] = fault.detail["stack"].slice(
        0,
        stackTrimIndex.index,
      );
    }
  }
  return fault;
 }
 // Need this for Node < 15
 process.on("unhandledRejection", (err) => {
  throw err;
 });
 process.on("uncaughtException", (err) => {
  const fault = errorToFault(err);
  jobs.forEach((jobId) => {
    process.send([jobId, fault, null]);
  });
  jobs.clear();
  process.disconnect();
 });
 process.on("message", (message) => {
  jobs.add(message[0]);
  if (!script) {
    const cwd = process.env["GENIEACS_EXT_DIR"];
    process.chdir(cwd);
    // eslint-disable-next-line @typescript-eslint/no-require-imports
    script = require(`${cwd}/${fileName}`);
  }
  const funcName = message[1][0];
  if (!script[funcName]) {
    const fault = {
      code: "ext",
      message: `No such function '${funcName}' in extension '${fileName}'`,
    };
    process.send([message[0], fault, null]);
    return;
  }
  script[funcName](message[1].slice(1), (err, res) => {
    if (!jobs.delete(message[0])) return;
    process.send([message[0], errorToFault(err), res]);
  });
 });
 // Ignore SIGINT
 process.on("SIGINT", () => {
  // Ignore
 });
@@ -0,0 +1,9 @@
 #!/usr/bin/env node
 var path = require('path');
 var fs = require('fs');
 var dir = path.resolve(path.dirname(fs.realpathSync(__filename)), '..');
 process.chdir(dir);
 cluster = require(path.resolve(dir, 'lib/cluster'));
 cluster.start('fs');
@@ -1,100 +0,0 @@
 import * as config from "../lib/config.ts";
 import * as logger from "../lib/logger.ts";
 import * as cluster from "../lib/cluster.ts";
 import * as server from "../lib/server.ts";
 import { listener } from "../lib/fs.ts";
 import * as db from "../lib/db/db.ts";
 import { version as VERSION } from "../package.json";
 logger.init("fs", VERSION);
 const SERVICE_ADDRESS = config.get("FS_INTERFACE") as string;
 const SERVICE_PORT = config.get("FS_PORT") as number;
 function exitWorkerGracefully(): void {
  setTimeout(exitWorkerUngracefully, 5000).unref();
  Promise.all([db.disconnect(), cluster.worker.disconnect()])
    .then(logger.close)
    .catch(exitWorkerUngracefully);
 }
 function exitWorkerUngracefully(): void {
  process.exit(1);
 }
 if (!cluster.worker) {
  const WORKER_COUNT = config.get("FS_WORKER_PROCESSES") as number;
  logger.info({
    message: `genieacs-fs starting`,
    pid: process.pid,
    version: VERSION,
  });
  cluster.start(WORKER_COUNT, SERVICE_PORT, SERVICE_ADDRESS);
  process.on("SIGINT", () => {
    logger.info({
      message: "Received signal SIGINT, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
  process.on("SIGTERM", () => {
    logger.info({
      message: "Received signal SIGTERM, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
 } else {
  const key = config.get("FS_SSL_KEY") as string;
  const cert = config.get("FS_SSL_CERT") as string;
  const options = {
    port: SERVICE_PORT,
    host: SERVICE_ADDRESS,
    ssl: key && cert ? { key, cert } : null,
    timeout: 30000,
  };
  // Need this for Node < 15
  process.on("unhandledRejection", (err) => {
    throw err;
  });
  process.on("uncaughtException", (err) => {
    if ((err as NodeJS.ErrnoException).code === "ERR_IPC_DISCONNECTED") return;
    logger.error({
      message: "Uncaught exception",
      exception: err,
      pid: process.pid,
    });
    server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
  });
  const initPromise = db
    .connect()
    .then(() => {
      server.start(options, listener);
    })
    .catch((err) => {
      setTimeout(() => {
        throw err;
      });
    });
  process.on("SIGINT", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
  process.on("SIGTERM", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
 }
@@ -0,0 +1,9 @@
 #!/usr/bin/env node
 var path = require('path');
 var fs = require('fs');
 var dir = path.resolve(path.dirname(fs.realpathSync(__filename)), '..');
 process.chdir(dir);
 cluster = require(path.resolve(dir, 'lib/cluster'));
 cluster.start('nbi');
@@ -1,107 +0,0 @@
 import * as config from "../lib/config.ts";
 import * as logger from "../lib/logger.ts";
 import * as cluster from "../lib/cluster.ts";
 import * as server from "../lib/server.ts";
 import { listener } from "../lib/nbi.ts";
 import * as db from "../lib/db/db.ts";
 import * as extensions from "../lib/extensions.ts";
 import { version as VERSION } from "../package.json";
 logger.init("nbi", VERSION);
 const SERVICE_ADDRESS = config.get("NBI_INTERFACE") as string;
 const SERVICE_PORT = config.get("NBI_PORT") as number;
 function exitWorkerGracefully(): void {
  setTimeout(exitWorkerUngracefully, 5000).unref();
  Promise.all([
    db.disconnect(),
    extensions.killAll(),
    cluster.worker.disconnect(),
  ])
    .then(logger.close)
    .catch(exitWorkerUngracefully);
 }
 function exitWorkerUngracefully(): void {
  void extensions.killAll().finally(() => {
    process.exit(1);
  });
 }
 if (!cluster.worker) {
  const WORKER_COUNT = config.get("NBI_WORKER_PROCESSES") as number;
  logger.info({
    message: `genieacs-nbi starting`,
    pid: process.pid,
    version: VERSION,
  });
  cluster.start(WORKER_COUNT, SERVICE_PORT, SERVICE_ADDRESS);
  process.on("SIGINT", () => {
    logger.info({
      message: "Received signal SIGINT, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
  process.on("SIGTERM", () => {
    logger.info({
      message: "Received signal SIGTERM, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
 } else {
  const key = config.get("NBI_SSL_KEY") as string;
  const cert = config.get("NBI_SSL_CERT") as string;
  const options = {
    port: SERVICE_PORT,
    host: SERVICE_ADDRESS,
    ssl: key && cert ? { key, cert } : null,
    timeout: 120000,
  };
  // Need this for Node < 15
  process.on("unhandledRejection", (err) => {
    throw err;
  });
  process.on("uncaughtException", (err) => {
    if ((err as NodeJS.ErrnoException).code === "ERR_IPC_DISCONNECTED") return;
    logger.error({
      message: "Uncaught exception",
      exception: err,
      pid: process.pid,
    });
    server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
  });
  const initPromise = db
    .connect()
    .then(() => {
      server.start(options, listener);
    })
    .catch((err) => {
      setTimeout(() => {
        throw err;
      });
    });
  process.on("SIGINT", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
  process.on("SIGTERM", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
 }
@@ -1,107 +0,0 @@
 import * as config from "../lib/config.ts";
 import * as logger from "../lib/logger.ts";
 import * as cluster from "../lib/cluster.ts";
 import * as server from "../lib/server.ts";
 import { listener } from "../lib/ui.ts";
 import * as extensions from "../lib/extensions.ts";
 import * as db from "../lib/db/db.ts";
 import { version as VERSION } from "../package.json";
 logger.init("ui", VERSION);
 const SERVICE_ADDRESS = config.get("UI_INTERFACE") as string;
 const SERVICE_PORT = config.get("UI_PORT") as number;
 function exitWorkerGracefully(): void {
  setTimeout(exitWorkerUngracefully, 5000).unref();
  Promise.all([
    db.disconnect(),
    extensions.killAll(),
    cluster.worker.disconnect(),
  ])
    .then(logger.close)
    .catch(exitWorkerUngracefully);
 }
 function exitWorkerUngracefully(): void {
  void extensions.killAll().finally(() => {
    process.exit(1);
  });
 }
 if (!cluster.worker) {
  const WORKER_COUNT = config.get("UI_WORKER_PROCESSES") as number;
  logger.info({
    message: `genieacs-ui starting`,
    pid: process.pid,
    version: VERSION,
  });
  cluster.start(WORKER_COUNT, SERVICE_PORT, SERVICE_ADDRESS);
  process.on("SIGINT", () => {
    logger.info({
      message: "Received signal SIGINT, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
  process.on("SIGTERM", () => {
    logger.info({
      message: "Received signal SIGTERM, exiting",
      pid: process.pid,
    });
    cluster.stop();
  });
 } else {
  const key = config.get("UI_SSL_KEY") as string;
  const cert = config.get("UI_SSL_CERT") as string;
  const options = {
    port: SERVICE_PORT,
    host: SERVICE_ADDRESS,
    ssl: key && cert ? { key, cert } : null,
    timeout: 120000,
  };
  // Need this for Node < 15
  process.on("unhandledRejection", (err) => {
    throw err;
  });
  process.on("uncaughtException", (err) => {
    if ((err as NodeJS.ErrnoException).code === "ERR_IPC_DISCONNECTED") return;
    logger.error({
      message: "Uncaught exception",
      exception: err,
      pid: process.pid,
    });
    server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
  });
  const initPromise = db
    .connect()
    .then(() => {
      server.start(options, async (req, res) => listener(req, res));
    })
    .catch((err) => {
      setTimeout(() => {
        throw err;
      });
    });
  process.on("SIGINT", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
  process.on("SIGTERM", () => {
    void initPromise.finally(() => {
      server.stop().then(exitWorkerGracefully).catch(exitWorkerUngracefully);
    });
  });
 }
@@ -1,5 +0,0 @@
 export const APP_JS = "app.js";
 export const APP_CSS = "app.css";
 export const ICONS_SVG = "icons.svg";
 export const LOGO_SVG = "logo.svg";
 export const FAVICON_PNG = "favicon.png";
@@ -1,396 +0,0 @@
 import path from "node:path";
 import fs from "node:fs";
 import { createHash } from "node:crypto";
 import { promisify } from "node:util";
 import { exec } from "node:child_process";
 import * as esbuild from "esbuild";
 import { optimize } from "svgo";
 import * as xmlParser from "../lib/xml-parser.ts";
 const fsAsync = {
  readdir: promisify(fs.readdir),
  readFile: promisify(fs.readFile),
  writeFile: promisify(fs.writeFile),
  copyFile: promisify(fs.copyFile),
  rename: promisify(fs.rename),
  chmod: promisify(fs.chmod),
  lstat: promisify(fs.lstat),
  exists: promisify(fs.exists),
  rmdir: promisify(fs.rmdir),
  unlink: promisify(fs.unlink),
  mkdir: promisify(fs.mkdir),
 };
 const execAsync = promisify(exec);
 const MODE = process.env["NODE_ENV"] || "production";
 const INPUT_DIR = process.cwd();
 const OUTPUT_DIR = path.join(INPUT_DIR, "dist");
 async function rmDir(dirPath: string): Promise<void> {
  if (!(await fsAsync.exists(dirPath))) return;
  const files = await fsAsync.readdir(dirPath);
  for (const file of files) {
    const filePath = path.join(dirPath, file);
    if ((await fsAsync.lstat(filePath)).isDirectory()) await rmDir(filePath);
    else await fsAsync.unlink(filePath);
  }
  await fsAsync.rmdir(dirPath);
 }
 // For lockfileVersion = 1
 function stripDevDeps(deps): void {
  if (!deps["dependencies"]) return;
  for (const [k, v] of Object.entries(deps["dependencies"])) {
    if (v["dev"]) delete deps["dependencies"][k];
    else stripDevDeps(v);
  }
  if (!Object.keys(deps["dependencies"]).length) delete deps["dependencies"];
 }
 // For lockfileVersion = 2
 function stripDevDeps2(deps): void {
  if (!deps["packages"]) return;
  for (const [k, v] of Object.entries(deps["packages"])) {
    delete v["devDependencies"];
    if (v["dev"]) delete deps["packages"][k];
  }
 }
 function xmlTostring(xml): string {
  const children = [];
  for (const c of xml.children || []) children.push(xmlTostring(c));
  return xml.name === "root" && xml.bodyIndex === 0
    ? children.join("")
    : `<${xml.name} ${xml.attrs}>${children.join("")}</${xml.name}>`;
 }
 function assetHash(buffer: Buffer | string): string {
  return createHash("md5").update(buffer).digest("hex").slice(0, 8);
 }
 const ASSETS = {} as {
  APP_JS?: string;
  APP_CSS?: string;
  ICONS_SVG?: string;
  LOGO_SVG?: string;
  FAVICON_PNG?: string;
 };
 const assetsPlugin = {
  name: "assets",
  setup(build) {
    build.onLoad({ filter: /\/build\/assets.ts$/ }, () => {
      const lines = Object.entries(ASSETS).map(
        ([k, v]) => `export const ${k} = ${JSON.stringify(v)};`,
      );
      return { contents: lines.join("\n") };
    });
  },
 } as esbuild.Plugin;
 const seedPlugin = {
  name: "seed",
  setup(build) {
    build.onLoad({ filter: /\/seed\// }, (args) => {
      if (args.with?.["type"] !== "text") return undefined;
      let contents = fs.readFileSync(args.path, "utf8");
      // Strip TypeScript directives that are only needed for type-checking
      contents = contents.replace(/^\s*\/\/\s*@ts-.*\n/gm, "");
      return { contents, loader: "text" };
    });
  },
 } as esbuild.Plugin;
 const packageDotJsonPlugin = {
  name: "packageDotJson",
  setup(build) {
    const sourcePath = path.join(INPUT_DIR, "package.json");
    build.onResolve({ filter: /\/package.json$/ }, (args) => {
      const p = path.join(args.resolveDir, args.path);
      if (p !== sourcePath) return undefined;
      return { path: path.join(OUTPUT_DIR, "package.json") };
    });
  },
 } as esbuild.Plugin;
 const inlineDepsPlugin = {
  name: "inlineDeps",
  setup(build) {
    const deps = ["espresso-iisojs", "@codemirror", "mithril", "yaml"];
    const depFiles = new Set();
    build.onResolve({ filter: /^[^.]/ }, async (args) => {
      if (args.pluginData === "inlineDeps") return undefined;
      if (
        depFiles.has(args.importer) ||
        deps.some((d) => args.path.startsWith(d))
      ) {
        const res = await build.resolve(args.path, {
          importer: args.importer,
          namespace: args.namespace,
          resolveDir: args.resolveDir,
          kind: args.kind,
          with: args.with,
          pluginData: "inlineDeps",
        });
        depFiles.add(res.path);
        return res;
      }
      return { sideEffects: false, external: true };
    });
  },
 } as esbuild.Plugin;
 function generateSymbol(id: string, svgStr: string): string {
  const xml = xmlParser.parseXml(svgStr);
  const svg = xml.children[0];
  const svgAttrs = xmlParser.parseAttrs(svg.attrs);
  let viewBox = "";
  for (const a of svgAttrs) {
    if (a.name === "viewBox") {
      viewBox = `viewBox="${a.value}"`;
      break;
    }
  }
  const symbolBody = xml.children[0].children
    .map((c) => xmlTostring(c))
    .join("");
  return `<symbol id="icon-${id}" ${viewBox}>${symbolBody}</symbol>`;
 }
 async function getBuildMetadata(): Promise<string> {
  const date = new Date().toISOString().slice(2, 10).replaceAll("-", "");
  const [commit, diff, newFiles] = await Promise.all([
    execAsync("git rev-parse HEAD"),
    execAsync("git diff HEAD"),
    execAsync("git ls-files --others --exclude-standard"),
  ]).then((res) => res.map((r) => r.stdout.trim()));
  if (!diff && !newFiles) return date + commit.slice(0, 4);
  const hash = createHash("md5");
  hash.update(commit).update(diff).update(newFiles);
  for (const file of newFiles.split("\n").filter((f) => f))
    hash.update(await fsAsync.readFile(file));
  return date + hash.digest("hex").slice(0, 4);
 }
 async function init(): Promise<void> {
  const [buildMetadata, packageJsonFile, npmShrinkwrapFile] = await Promise.all(
    [
      getBuildMetadata(),
      fsAsync.readFile(path.join(INPUT_DIR, "package.json")),
      fsAsync.readFile(path.join(INPUT_DIR, "npm-shrinkwrap.json")),
    ],
  );
  const packageJson = JSON.parse(packageJsonFile.toString());
  delete packageJson["devDependencies"];
  delete packageJson["private"];
  delete packageJson["scripts"];
  packageJson["version"] = `${packageJson["version"]}+${buildMetadata}`;
  const npmShrinkwrap = JSON.parse(npmShrinkwrapFile.toString());
  npmShrinkwrap["version"] = packageJson["version"];
  stripDevDeps(npmShrinkwrap);
  stripDevDeps2(npmShrinkwrap);
  await rmDir(OUTPUT_DIR);
  await fsAsync.mkdir(OUTPUT_DIR);
  await Promise.all([
    fsAsync.mkdir(path.join(OUTPUT_DIR, "bin")),
    fsAsync.mkdir(path.join(OUTPUT_DIR, "public")),
    fsAsync.writeFile(
      path.join(OUTPUT_DIR, "package.json"),
      JSON.stringify(packageJson, null, 2),
    ),
    fsAsync.writeFile(
      path.join(OUTPUT_DIR, "npm-shrinkwrap.json"),
      JSON.stringify(npmShrinkwrap, null, 2),
    ),
  ]);
 }
 async function copyStatic(): Promise<void> {
  const files = [
    "LICENSE",
    "README.md",
    "CHANGELOG.md",
    "public/logo.svg",
    "public/favicon.png",
  ];
  const [logo, favicon] = await Promise.all([
    fsAsync.readFile(path.join(INPUT_DIR, "public/logo.svg")),
    fsAsync.readFile(path.join(INPUT_DIR, "public/favicon.png")),
  ]);
  ASSETS.LOGO_SVG = `logo-${assetHash(logo)}.svg`;
  ASSETS.FAVICON_PNG = `favicon-${assetHash(favicon)}.png`;
  const filenames = {} as Record<string, string>;
  filenames["public/logo.svg"] = path.join("public", ASSETS.LOGO_SVG);
  filenames["public/favicon.png"] = path.join("public", ASSETS.FAVICON_PNG);
  await Promise.all(
    files.map((f) =>
      fsAsync.copyFile(
        path.join(INPUT_DIR, f),
        path.join(OUTPUT_DIR, filenames[f] || f),
      ),
    ),
  );
 }
 async function generateCss(): Promise<void> {
  const tailwindPlugin = {
    name: "tailwind",
    setup(build) {
      build.onLoad({ filter: /\/ui\/css\/app.css$/ }, async (args) => {
        const res = await execAsync(`npx @tailwindcss/cli -i ${args.path}`);
        return { loader: "css", contents: res.stdout };
      });
    },
  } as esbuild.Plugin;
  const res = await esbuild.build({
    bundle: true,
    absWorkingDir: INPUT_DIR,
    minify: MODE === "production",
    sourcemap: "linked",
    sourcesContent: false,
    entryPoints: ["ui/css/app.css"],
    entryNames: "[dir]/[name]-[hash]",
    outfile: path.join(OUTPUT_DIR, "public/app.css"),
    plugins: [tailwindPlugin],
    loader: {
      ".woff2": "dataurl",
    },
    target: ["chrome111", "safari16.4", "firefox128"],
    metafile: true,
  });
  for (const [k, v] of Object.entries(res.metafile.outputs)) {
    if (v.entryPoint === "ui/css/app.css") {
      ASSETS.APP_CSS = path.relative(
        path.join(OUTPUT_DIR, "public"),
        path.join(INPUT_DIR, k),
      );
      break;
    }
  }
 }
 async function generateBackendJs(): Promise<void> {
  const services = [
    "genieacs-cwmp",
    "genieacs-ext",
    "genieacs-nbi",
    "genieacs-fs",
    "genieacs-ui",
  ];
  await esbuild.build({
    bundle: true,
    absWorkingDir: INPUT_DIR,
    minify: MODE === "production",
    sourcemap: "inline",
    sourcesContent: false,
    platform: "node",
    target: "node12.13.0",
    packages: "external",
    banner: { js: "#!/usr/bin/env node" },
    entryPoints: services.map((s) => `bin/${s}.ts`),
    outdir: path.join(OUTPUT_DIR, "bin"),
    plugins: [packageDotJsonPlugin, assetsPlugin, seedPlugin],
  });
  for (const bin of services) {
    const p = path.join(OUTPUT_DIR, "bin", bin);
    await fsAsync.rename(`${p}.js`, p);
    // Mark as executable
    const mode = (await fsAsync.lstat(p)).mode;
    await fsAsync.chmod(p, mode | 73);
  }
 }
 async function generateFrontendJs(): Promise<void> {
  const res = await esbuild.build({
    bundle: true,
    absWorkingDir: INPUT_DIR,
    splitting: true,
    minify: MODE === "production",
    sourcemap: "linked",
    sourcesContent: false,
    platform: "browser",
    format: "esm",
    target: ["chrome111", "safari16.4", "firefox128"],
    entryPoints: ["ui/app.ts"],
    entryNames: "[dir]/[name]-[hash]",
    outdir: path.join(OUTPUT_DIR, "public"),
    plugins: [packageDotJsonPlugin, inlineDepsPlugin, assetsPlugin],
    metafile: true,
  });
  for (const [k, v] of Object.entries(res.metafile.outputs)) {
    for (const imp of v.imports)
      if (imp.external) throw new Error(`External import found: ${imp.path}`);
    if (v.entryPoint === "ui/app.ts") {
      ASSETS.APP_JS = path.relative(
        path.join(OUTPUT_DIR, "public"),
        path.join(INPUT_DIR, k),
      );
    }
  }
 }
 async function generateIconsSprite(): Promise<void> {
  const symbols = [] as string[];
  const iconsDir = path.join(INPUT_DIR, "ui/icons");
  for (const file of await fsAsync.readdir(iconsDir)) {
    const id = path.parse(file).name;
    const filePath = path.join(iconsDir, file);
    const src = (await fsAsync.readFile(filePath)).toString();
    const { data } = await optimize(src, {
      plugins: [
        {
          name: "preset-default",
          params: {
            overrides: {
              removeViewBox: false,
            },
          },
        },
      ],
    });
    symbols.push(generateSymbol(id, data));
  }
  const data = `<svg xmlns="http://www.w3.org/2000/svg">${symbols.join(
    "",
  )}</svg>`;
  ASSETS.ICONS_SVG = `icons-${assetHash(data)}.svg`;
  await fsAsync.writeFile(
    path.join(OUTPUT_DIR, "public", ASSETS.ICONS_SVG),
    data,
  );
 }
 init()
  .then(() =>
    Promise.all([
      Promise.all([generateIconsSprite(), copyStatic()]).then(
        generateFrontendJs,
      ),
      generateCss(),
    ]).then(generateBackendJs),
  )
  .catch((err) => {
    process.stderr.write(err.stack + "\n");
  });
@@ -1,18 +0,0 @@
 #!/bin/bash
 cd "$(dirname "$0")"
 declare -A FONTS=(
  [InterVariable]=https://rsms.me/inter/font-files/InterVariable.woff2
  [InterVariable-Italic]=https://rsms.me/inter/font-files/InterVariable-Italic.woff2
  [RobotoMono]=https://raw.githubusercontent.com/googlefonts/RobotoMono/main/fonts/variable/RobotoMono%5Bwght%5D.ttf
 )
 for name in "${!FONTS[@]}"
 do
  url="${FONTS[$name]}"
  echo "Downloading $url"
  TMP=$(mktemp)
  curl "$url" -s --output "$TMP"
  pyftsubset "$TMP" --output-file="../ui/css/$name.woff2" --flavor=woff2 --unicodes="U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD" --layout-features+="tnum"
 done
@@ -1,55 +0,0 @@
 import { exec } from "node:child_process";
 import { promisify } from "node:util";
 const execPromise = promisify(exec);
 async function runEslint(): Promise<string> {
  const CMD =
    "eslint 'bin/*.ts' 'lib/**/*.ts' 'ui/**/*.ts' 'test/**/*.ts' 'build/**/*.ts' 'seed/*'";
  const env = {
    ...(process.stdout.isTTY && { FORCE_COLOR: "1" }),
    ...process.env,
  };
  try {
    const { stdout, stderr } = await execPromise(CMD, { env });
    if (stderr) throw new Error(stderr);
    return stdout;
  } catch (err) {
    if (err.killed || err.signal || err.stderr || err.code !== 1) throw err;
    return err.stdout;
  }
 }
 async function runTsc(): Promise<string> {
  const CMD = "tsc --noEmit && tsc -p seed/tsconfig.json";
  const env = {
    ...(process.stdout.isTTY && { FORCE_COLOR: "1" }),
    ...process.env,
  };
  const { stdout, stderr } = await execPromise(CMD, { env });
  if (stderr) throw new Error(stderr);
  return stdout;
 }
 async function runPrettier(): Promise<string> {
  const CMD = "prettier --prose-wrap always --write .";
  const env = {
    ...(process.stdout.isTTY && { FORCE_COLOR: "1" }),
    ...process.env,
  };
  const { stdout, stderr } = await execPromise(CMD, { env });
  if (stderr) throw new Error(stderr);
  return stdout;
 }
 async function runAll(): Promise<void> {
  const prom1 = runPrettier();
  const prom2 = runEslint();
  const prom3 = runTsc();
  console.log(await prom1);
  console.log(await prom2);
  console.log(await prom3);
 }
 runAll().catch(console.error);
@@ -1,180 +0,0 @@
 personal_ws-1.1 en 0
 genieacs
 GenieACS
 javascript
 sudo
 config
 cwmp
 CPE
 systemctl
 nbi
 CWMP
 NBI
 ACS
 SSL
 xsd
 args
 DeviceID
 GENIEACS
 env
 param
 arg
 AUTH
 igd
 TCP
 toctree
 auth
 EnvironmentFile
 JSON
 json
 parameterValues
 setParameterValues
 stderr
 stdout
 attr
 cfg
 CPEs
 ExecStart
 faq
 FileType
 GenieACS's
 JWT
 maxdepth
 npm
 objectName
 refreshObject
 SerialNumber
 systemd
 usr
 WantedBy
 yaml
 chown
 Config
 getParameterValues
 https
 InternetGatewayDevice
 mkdir
 OUI
 sql
 TODO
 UDP
 addObject
 AGPLv
 const
 cpe
 cron
 deleteObject
 FactoryReset
 factoryReset
 FileName
 fileType
 Github
 hostname
 HTTPS
 jwt
 latlong
 NPM
 oui
 parameterNames
 ProductClass
 productClass
 repo
 SSID
 sublicense
 TLS
 WANIPConnection
 WPA
 YourProvisionName
 abc
 Abdulla
 accessors
 AddObject
 AddressingType
 api
 APIs
 chmod
 cwmp's
 dateext
 DATETIME
 dateTime
 datetime
 declaratively
 delaycompress
 DeleteObject
 deviceId
 DHCP
 dir
 distro
 equisatisfiable
 ExternalIPAddress
 failover
 favor
 func
 GetParameterAttributes
 getPassword
 gte
 gui
 hexBinary
 Hils
 HOSTNAME
 init
 journalctl
 journald
 LastFileName
 LastFileType
 lastInform
 LIBXMLJS
 libxmljs
 literalinclude
 lockfile
 logrotate
 lte
 mipsbe
 normalizers
 pre
 quickstart
 README
 redis
 reimplemented
 RequestDownload
 RESTful
 rollout
 sandboxed
 scalable
 SetParameterAttributes
 SetParamteerValues
 SIGINT
 SIGTERM
 skipWritableCheck
 stringify
 subdirectory
 TargetFileName
 useradd
 VirtualParameters
 vparam
 xml
 YAML
 Zaid
 cacheExpire
 iss
 http
 statusCode
 rawData
 pos
 Brotli
 CSV
 hostInfo
 IPv
 downloadSuccessOnTimeout
 TransferComplete
 nginx
 ENCODEURICOMPONENT
 pageSize
 skipRootGpn
 GPN
 NaN
 CIDR
 XMPP
 PEM
 JID
 unindexed
@@ -1,12 +0,0 @@
 #! /bin/sh
 cd "$(dirname "$0")"
 FILES=`ls ../docs/*.rst ../docs/*.js ../*.md`
 for FILE in $FILES
 do
    echo $FILE
    cat $FILE | aspell list --lang=en --add-extra-dicts=./spellcheck-dict.pws --ignore 2
    echo
 done
@@ -0,0 +1,4 @@
 exports.connectionRequest = function(deviceId) {
  // return username/password pair for a given device
  return ["", ""];
 }
@@ -0,0 +1,15 @@
 {
  "MONGODB_CONNECTION_URL" : "mongodb://127.0.0.1/genieacs",
  "REDIS_PORT" : "6379",
  "REDIS_HOST" : "127.0.0.1",
  "CWMP_INTERFACE" : "0.0.0.0",
  "CWMP_PORT" : 7547,
  "CWMP_SSL" : false,
  "NBI_INTERFACE" : "0.0.0.0",
  "NBI_PORT" : 7557,
  "FS_INTERFACE" : "0.0.0.0",
  "FS_PORT" : 7567,
  "FS_IP" : "192.168.0.1",
  "LOG_INFORMS" : true,
  "DEBUG" : false
 }
@@ -0,0 +1,4 @@
 {
  "seowon_frequencies" : "/^[0-9]+\\-SWU\\%2D9200\\-[0-9A-Z]+$/",
  "greenpacket_frequencies" : "/^[0-9A-Z]+\\-DX350\\-[0-9A-Z]+$/"
 }
@@ -0,0 +1,210 @@
 /*
 * Copyright 2013 Fanoos Telecom
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */
 var net = require('net');
 var PORT = 23;
 var USERNAME = 'mt7109';
 var PASSWORD = 'wimax';
 var TIMEOUT = 5000;
 var FREQUENCIES_REGEX = /([0-9,]+)/
 var BANDWIDTHS_REGEX = /[0-9]+\-([0-9,]+)/
 var DEFAULT_BW_REGEX = /([0-9]+)\-[0-9,]+/
 var telnetConnect = function(host, callback) {
  var client = net.connect(PORT, host, function() {
    // enter username
    telnetExecute(USERNAME, client, "Password: ", function(err, response) {
      if (err) return callback(err);
      // enter password
      telnetExecute(PASSWORD, client, "\n# ", function(err, response) {
        return callback(err);
      });
    });
  });
  return client;
 }
 var telnetExecute = function(command, connection, prompt, callback) {
  var response = '';
  command += "\n";
  var listener = function(chunk) {
    response += chunk.toString();
    if (response.slice(0 - prompt.length) == prompt) {
      connection.removeListener('data', listener);
      return callback(null, response.slice(command.length, 0 - prompt.length));
    }
  };
  var end = function() {
    connection.removeListener('end', end);
    return callback(null, response.slice(command.length));
  };
  if (prompt)
    connection.on('data', listener);
  else
    connection.on('end', end);
  if (command)
    connection.write(command);
 }
 var getDeviceIp = function(deviceId, callback) {
  var db = require('../../lib/db');
  var URL = require('url');
  db.devicesCollection.findOne({_id : deviceId}, {'InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value' : 1}, function(err, device) {
    var connectionRequestUrl = device.InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value;
    var url = URL.parse(connectionRequestUrl);
    return callback(url.hostname);
  });
 };
 exports.init = function(deviceId, args, callback) {
  return exports.get(deviceId, args, callback);
 }
 exports.get = function(deviceId, args, callback) {
  var currentFrequencies = [];
  var currentBandwidths = [];
  var defaultBandwidth = '';
  var final = [];
  getDeviceIp(deviceId, function(ip) {
    var client = telnetConnect(ip, function(err) {
      // get frequencies
      telnetExecute("sncfg get WMX_FREQ_LIST", client, "\n# ", function(err, response) {
        if (err) return callback(err);
        currentFrequencies = FREQUENCIES_REGEX.exec(response)[1].split(',')
        // get bandwidths
        telnetExecute("sncfg get WMX_FREQ_BANDWDITH_FL", client, "\n# ", function(err, response) {
          if (err) return callback(err);
          currentBandwidths = BANDWIDTHS_REGEX.exec(response)[1].split(',')
          defaultBandwidth = DEFAULT_BW_REGEX.exec(response)[1]
          // generate freq/bw pairs string
          for (var i = 0; i < currentFrequencies.length; i++) {
            final.push(currentFrequencies[i] + ' ' + currentBandwidths[i])
          }
          // log out
          telnetExecute("exit", client, null, function(err, response) {
            client.end();
            return callback(err, defaultBandwidth + '-' + final.join(','));
          });
        });
      });
    });
    client.on("error", function(err) {
      client.destroy();
      callback(err);
    });
    client.setTimeout(TIMEOUT, function() {
      client.destroy();
      callback("timeout");
    });
  });
 };
 var parsePairs = function(pairsString) {
  var a = pairsString.split(',');
  var f = [];
  var b = [];
  var l = [];
  for (i in a) {
    var sp = a[i].trim().split(' ');
    f.push(parseInt(sp[0]));
    b.push(parseFloat(sp[1]));
  }
  l.push(f.join(','));
  l.push(b.join(','));
  return l;
 }
 exports.set = function(deviceId, args, callback) {
  executeSet(deviceId, args, ['set'], callback);
 };
 exports.dset = function(deviceId, args, callback) {
  executeSet(deviceId, args, ['dset'], callback);
 };
 exports.setdset = function(deviceId, args, callback) {
  executeSet(deviceId, args, ['dset', 'set'], callback);
 };
 var executeSet = function(deviceId, args, method, callback) {
  var tmp = args.split('-');
  var defaultBandwidth = tmp[0];
  var pairs = parsePairs(tmp[1]);
  var frequenciesString = pairs[0];
  var bandwidthsString = pairs[1];
  getDeviceIp(deviceId, function(ip) {
    var client = telnetConnect(ip, function(err) {
      var telnet_cmd = "sncfg " + method[0];
      telnetExecute(telnet_cmd + " WMX_FREQ_BANDWDITH_FL '" + defaultBandwidth + "-" + bandwidthsString + "'", client, "\n# ", function(err, response) {
        if (err) return callback(err);
        telnetExecute(telnet_cmd + " WMX_FREQ_LIST '" + frequenciesString + "'", client, "\n# ", function(err, response) {
          if (err) return callback(err);
          if (method.length > 1) {
            telnet_cmd = "sncfg " + method[1];
            telnetExecute(telnet_cmd + " WMX_FREQ_BANDWDITH_FL '" + defaultBandwidth + "-" + bandwidthsString + "'", client, "\n# ", function(err, response) {
              if (err) return callback(err);
              telnetExecute(telnet_cmd + " WMX_FREQ_LIST '" + frequenciesString + "'", client, "\n# ", function(err, response) {
                if (err) return callback(err);
                telnetExecute("sncfg commit", client, "\n# ", function(err, response) {
                  if (err) return callback(err);
                  // log out
                  telnetExecute("exit", client, null, function(err, response) {
                    client.end();
                    return callback(null, args);
                  });
                });
              });
            });
          }
          else {
            telnetExecute("sncfg commit", client, "\n# ", function(err, response) {
            if (err) return callback(err);
              // log out
              telnetExecute("exit", client, null, function(err, response) {
                client.end();
                return callback(null, args);
              });
            });
          }
        });
      });
    });
    client.on("error", function(err) {
      client.destroy();
      callback(err);
    });
    client.setTimeout(TIMEOUT, function() {
      client.destroy();
      callback("timeout");
    });
  });
 };
@@ -0,0 +1,230 @@
 /*
 * Copyright 2013 Fanoos Telecom
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */
 // Change frequency plans for Seowon devices.
 // Args format: 3532500 5000,3537500 5000,3542500 5000,3547500 5000,3552500 5000,3557500 5000,3505000 10000,3515000 10000,3525000 10000,3535000 10000,3545000 10000,3555000 10000
 var net = require('net');
 var PORT = 23;
 var USERNAME = 'admin';
 var PASSWORD = 'admin';
 var TIMEOUT = 5000;
 var FREQUENCIES_REGEX = /\s*([\d]+)\.\s*Frequency\s*:\s*([\d]+)\s\(KHz\),\s*Bandwidth\s*:\s*([\d\.]+)\s*\(MHz\)/gm
 var telnetConnect = function(host, callback) {
  var client = net.connect(PORT, host, function() {
    // enter username
    telnetExecute(USERNAME, client, "Password: ", function(err, response) {
      if (err) return callback(err);
      // enter password
      telnetExecute(PASSWORD, client, "MT7109> ", function(err, response) {
        return callback(err);
      });
    });
  });
  return client;
 }
 var telnetExecute = function(command, connection, prompt, callback) {
  var response = '';
  command += "\n";
  var listener = function(chunk) {
    response += chunk.toString();
    if (response.slice(0 - prompt.length) == prompt) {
      connection.removeListener('data', listener);
      return callback(null, response.slice(command.length, 0 - prompt.length));
    }
  };
  var end = function() {
    connection.removeListener('end', end);
    return callback(null, response.slice(command.length));
  };
  if (prompt)
    connection.on('data', listener);
  else
    connection.on('end', end);
  if (command)
    connection.write(command);
 }
 var parseFrequencies = function(frequenciesString) {
  var a = frequenciesString.split(',');
  var l = [];
  for (i in a) {
    var sp = a[i].trim().split(' ');
    var frequency = parseInt(sp[0]);
    var bandwidth = parseFloat(sp[1]);
    l.push([frequency, bandwidth]);
  }
  return l;
 }
 var findFrequency = function(haystack, needle) {
  for (i in haystack) {
    if (haystack[i][0] == needle[0] && haystack[i][1] == needle[1])
      return i;
  }
  return -1;
 }
 var getDeviceIp = function(deviceId, callback) {
  var db = require('../../lib/db');
  var URL = require('url');
  db.devicesCollection.findOne({_id : deviceId}, {'InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value' : 1}, function(err, device) {
    var connectionRequestUrl = device.InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value;
    var url = URL.parse(connectionRequestUrl);
    return callback(url.hostname);
  });
 };
 exports.init = function(deviceId, args, callback) {
  return exports.get(deviceId, args, callback);
 };
 exports.get = function(deviceId, args, callback) {
  var currentFrequencies = [];
  getDeviceIp(deviceId, function(ip) {
    var client = telnetConnect(ip, function(err) {
      // enter privileged mode
      telnetExecute("enable", client, "MT7109# ", function(err, response) {
        if (err) return callback(err);
        // show frequencies
        telnetExecute("show wmx freq", client, "MT7109# ", function(err, response) {
          if (err) return callback(err);
          while (f = FREQUENCIES_REGEX.exec(response)) {
            var frequency = parseInt(f[2]);
            var bandwidth = Math.floor(parseFloat(f[3]) * 1000);
            var freqStr = String(frequency) + ' ' + String(bandwidth);
            currentFrequencies.push(freqStr);
          }
          // log out
          telnetExecute("logout", client, null, function(err, response) {
            client.end();
            return callback(err, currentFrequencies.join(','));
          });
        });
      });
    });
    client.on("error", function(err) {
      client.destroy();
      callback(err);
    });
    client.setTimeout(TIMEOUT, function() {
      client.destroy();
      callback("timeout");
    });
  });
 };
 exports.set = function(deviceId, args, callback) {
  var newFrequencies = parseFrequencies(args);
  var addFrequencies = [];
  var delFrequencies = [];
  getDeviceIp(deviceId, function(ip) {
    var client = telnetConnect(ip, function(err) {
      // enter privileged mode
      telnetExecute("enable", client, "MT7109# ", function(err, response) {
        if (err) return callback(err);
        // show frequencies
        telnetExecute("show wmx freq", client, "MT7109# ", function(err, response) {
          if (err) return callback(err);
          while (f = FREQUENCIES_REGEX.exec(response)) {
            var index = parseInt(f[1]);
            var frequency = parseInt(f[2]);
            var bandwidth = Math.floor(parseFloat(f[3]));
            var i = findFrequency(newFrequencies, [frequency, bandwidth * 1000]);
            if (i == -1)
              delFrequencies.push(index);
            else
              newFrequencies.splice(i, 1);
          }
          for (f in newFrequencies)
            addFrequencies.push(newFrequencies[f]);
          if (addFrequencies.length == 0 && delFrequencies.length == 0) {
            // log out
            telnetExecute("logout", client, null, function(err, response) {
              client.end();
              return callback(err, args);
            });
            return;
          }
          telnetExecute("wimax", client, "MT7109 (WiMax)# ", function(err, response) {
            if (err) return callback(err);
            var add, del;
            del = function() {
              if (delFrequencies.length == 0) {
                add();
                return;
              }
              var i = delFrequencies.pop();
              telnetExecute("wmx freq del " + i, client, "MT7109 (WiMax)# ", function(err, response) {
                if (err) return callback(err);
                del();
              });
            };
            add = function() {
              if (addFrequencies.length == 0) {
                // commit
                telnetExecute("commit", client, "MT7109 (WiMax)# ", function(err, response) {
                  if (err) return callback(err);
                  // log out
                  telnetExecute("logout", client, null, function(err, response) {
                    client.end();
                    return callback(null, args);
                  });
                });
                return;
              }
              var i = addFrequencies.pop();
              telnetExecute("wmx freq add " + i[0] + ' ' + i[1]/1000, client, "MT7109 (WiMax)# ", function(err, response) {
                if (err) return callback(err);
                add();
              });
            };
            del();
          });
        });
      });
    });
    client.on("error", function(err) {
      client.destroy();
      callback(err);
    });
    client.setTimeout(TIMEOUT, function() {
      client.destroy();
      callback("timeout");
    });
  });
 };
@@ -0,0 +1,27 @@
 {
  "_lastInform" : {"alias" : "summary.lastInform", "type" : "date"},
  "_lastBoot" : {"alias" : "summary.lastBoot", "type" : "date"},
  "_lastBootstrap" : {"alias" : "summary.lastBootstrap", "type" : "date"},
  "_deviceId._SerialNumber" : {"alias" : "summary.serialNumber"},
  "_deviceId._ProductClass" : {"alias" : "summary.productClass"},
  "_deviceId._OUI" : {"alias" : "summary.oui"},
  "_deviceId._Manufacturer" : {"alias" : "summary.manufacturer"},
  "InternetGatewayDevice.DeviceInfo.HardwareVersion" : {"alias" : "summary.hardwareVersion"},
  "InternetGatewayDevice.DeviceInfo.SoftwareVersion" : {"alias" : "summary.softwareVersion"},
  "Device.DeviceInfo.HardwareVersion" : {"alias" : "summary.hardwareVersion"},
  "Device.DeviceInfo.SoftwareVersion" : {"alias" : "summary.softwareVersion"},
  "InternetGatewayDevice.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANIPConnection./^\\d+$/.MACAddress" : {"alias" : "summary.mac", "type" : "mac"},
  "InternetGatewayDevice.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANIPConnection./^\\d+$/.ExternalIPAddress" : {"alias" : "summary.ip"},
  "InternetGatewayDevice.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANPPPConnection./^\\d+$/.MACAddress" : {"alias" : "summary.mac", "type" : "mac"},
  "InternetGatewayDevice.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANPPPConnection./^\\d+$/.ExternalIPAddress" : {"alias" : "summary.ip"},
  "Device.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANIPConnection./^\\d+$/.MACAddress" : {"alias" : "summary.mac", "type" : "mac"},
  "Device.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANIPConnection./^\\d+$/.ExternalIPAddress" : {"alias" : "summary.ip"},
  "Device.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANPPPConnection./^\\d+$/.MACAddress" : {"alias" : "summary.mac", "type" : "mac"},
  "Device.WANDevice./^\\d+$/.WANConnectionDevice./^\\d+$/.WANPPPConnection./^\\d+$/.ExternalIPAddress" : {"alias" : "summary.ip"},
  "InternetGatewayDevice.LANDevice./^\\d+$/.WLANConfiguration./^\\d+$/.SSID" : {"alias" : "summary.wlanSsid"},
  "InternetGatewayDevice.LANDevice./^\\d+$/.WLANConfiguration./^\\d+$/.PreSharedKey.1.KeyPassphrase" : {"alias" : "summary.wlanPassphrase"},
  "Device.LANDevice./^\\d+$/.WLANConfiguration./^\\d+$/.SSID" : {"alias" : "summary.wlanSsid"},
  "Device.LANDevice./^\\d+$/.WLANConfiguration./^\\d+$/.PreSharedKey.1.KeyPassphrase" : {"alias" : "summary.wlanPassphrase"}
 }
@@ -1,134 +0,0 @@
 services:
  # ── MongoDB ──────────────────────────────────────────────────────────────
  mongo:
    image: mongo:6
    container_name: genieacs-mongo
    restart: unless-stopped
    volumes:
      - mongo_data:/data/db
      - mongo_config:/data/configdb
    networks:
      - genieacs
    healthcheck:
      test: ["CMD", "mongosh", "--quiet", "--eval", "db.adminCommand('ping').ok"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s
  # ── CWMP — TR-069 protocol handler (porta 7547) ──────────────────────────
  cwmp:
    build:
      context: .
      dockerfile: Dockerfile
      target: runtime
    image: genieacs-unifi:latest
    container_name: genieacs-cwmp
    restart: unless-stopped
    command: ["node", "bin/genieacs-cwmp"]
    ports:
      - "7547:7547"
    env_file: .env
    environment:
      GENIEACS_MONGODB_CONNECTION_URL: mongodb://mongo:27017/genieacs
    depends_on:
      mongo:
        condition: service_healthy
    volumes:
      - ext_dir:/opt/genieacs/config/ext
      - logs:/opt/genieacs/logs
    networks:
      - genieacs
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:7547/ || true"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 15s
  # ── NBI — Northbound REST API (porta 7557) ───────────────────────────────
  nbi:
    image: genieacs-unifi:latest
    container_name: genieacs-nbi
    restart: unless-stopped
    command: ["node", "bin/genieacs-nbi"]
    ports:
      - "7557:7557"
    env_file: .env
    environment:
      GENIEACS_MONGODB_CONNECTION_URL: mongodb://mongo:27017/genieacs
    depends_on:
      mongo:
        condition: service_healthy
      cwmp:
        condition: service_started
    volumes:
      - ext_dir:/opt/genieacs/config/ext
      - logs:/opt/genieacs/logs
    networks:
      - genieacs
  # ── FS — File Server / GridFS (porta 7567) ───────────────────────────────
  fs:
    image: genieacs-unifi:latest
    container_name: genieacs-fs
    restart: unless-stopped
    command: ["node", "bin/genieacs-fs"]
    ports:
      - "7567:7567"
    env_file: .env
    environment:
      GENIEACS_MONGODB_CONNECTION_URL: mongodb://mongo:27017/genieacs
    depends_on:
      mongo:
        condition: service_healthy
      cwmp:
        condition: service_started
    volumes:
      - ext_dir:/opt/genieacs/config/ext
      - logs:/opt/genieacs/logs
    networks:
      - genieacs
  # ── UI — Web Interface (porta 3000) ──────────────────────────────────────
  ui:
    image: genieacs-unifi:latest
    container_name: genieacs-ui
    restart: unless-stopped
    command: ["node", "bin/genieacs-ui"]
    ports:
      - "3000:3000"
    env_file: .env
    environment:
      GENIEACS_MONGODB_CONNECTION_URL: mongodb://mongo:27017/genieacs
    depends_on:
      mongo:
        condition: service_healthy
      nbi:
        condition: service_started
    volumes:
      - ext_dir:/opt/genieacs/config/ext
      - logs:/opt/genieacs/logs
    networks:
      - genieacs
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://localhost:3000/ || exit 1"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
 volumes:
  mongo_data:
    driver: local
  mongo_config:
    driver: local
  ext_dir:
    driver: local
  logs:
    driver: local
 networks:
  genieacs:
    driver: bridge
@@ -1,18 +0,0 @@
 # Read the Docs configuration file for Sphinx projects
 # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
 version: 2
 build:
  os: ubuntu-22.04
  tools:
    python: "3.12"
 sphinx:
  configuration: docs/conf.py
 formats: all
 python:
  install:
    - requirements: docs/requirements.txt
@@ -1,64 +0,0 @@
 .. _administration-faq:
 Administration FAQ
 ==================
 .. _administration-faq-duplicate-log-entries:
 Duplicate log entries when using :func:`log()` function
 -------------------------------------------------------
 Because GenieACS uses a full fledged scripting language for device
 configuration, the only way to guarantee that it has satisfied the 'desired
 state' is by repeatedly executing the script until there's no more
 discrepancies with the current device state. Though it may seem like this will
 cause duplicate requests going to the device, this isn't actually the case
 because device configuration are stated declaratively and that the scripts
 themselves are pure functions in the context of a session (e.g. Date.now()
 always returns the same value within the session).
 To illustrate with an example, consider the following script:
 .. code:: javascript
  log("Executing script");
  declare("Device.param", null, {value: 1});
  commit();
  declare("Device.param", null, {value: 2});
 This will set the value of the 'Device.param' to 1, then to 2. Then as the
 script is run again the value is set back to 1 and so on. A stable state will
 never be reached so GenieACS will execute the script a few times until it gives
 up and throws a fault. This is an edge case that should be avoided. A more
 typical case is where the script is run once or twice. Essentially if an
 execution doesn't result in any request to the CPE or a change in the data
 model then a stable state is deemed to have been reached.
 Configurations not pushed to device after factory reset
 ---------------------------------------------------------
 After a device is reset to its factory default state, the cached data model in
 GenieACS's database needs to be invalidated to force rediscovery. Ensure the
 following lines are called on ``0 BOOTSTRAP`` event:
 .. code:: javascript
  const now = Date.now();
  // Clear cached data model to force a refresh
  clear("Device", now);
  clear("InternetGatewayDevice", now);
 Most device parameters are missing
 ----------------------------------
 For performance reasons (server, client, and network), GenieACS by default only
 fetches parts of the data model that are necessary to satisfy the declarations
 in your provision scripts. Create declarations for any parameters you need
 fetched by default.
 If you're unsure and want to explore the available parameters exposed by the
 device, refresh the root parameter (e.g. ``InternetGatewayDevice``) from
 GenieACS's UI. You typically only need to do that one time for a given CPE
 model.
@@ -1,538 +0,0 @@
 API Reference
 =============
 GenieACS exposes a rich RESTful API through its NBI component. This document
 serves as a reference for the available APIs.
 This API makes use of MongoDB's query language in some of its endpoints. Refer
 to MongoDB's documentation for details.
 .. note::
  The examples below use ``curl`` command for simplicity and ease of testing.
  Query parameters are URL-encoded, but the original pre-encoding values are
  shown for reference. These examples assume genieacs-nbi is running locally
  and listening on the default NBI port (7557).
 .. warning::
  A common pitfall is not properly percent-encoding special characters in the
  device ID or query in the URL.
 Endpoints
 ---------
 GET /\<collection\>/?query=\<query\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Search for records in the database (e.g. devices, tasks, presets, files).
 Returns a JSON representation of all items in the given collection that match
 the search criteria.
 *collection*: The data collection to search. Could be one of: tasks, devices,
 presets, objects.
 *query*: Search query. Refer to MongoDB queries for reference.
 Examples
 ^^^^^^^^
 - Find a device by its ID:
 .. code:: javascript
  query = {"_id": "202BC1-BM632w-000000"}
 .. code:: bash
  curl -i 'http://localhost:7557/devices/?query=%7B%22_id%22%3A%22202BC1-BM632w-000000%22%7D'
 - Find a device by its MAC address:
 .. code:: javascript
  query = {
    "InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.1.MACAddress": "20:2B:C1:E0:06:65"
  }
 .. code:: bash
  curl -i 'http://localhost:7557/devices/?query=%7B%22InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.1.MACAddress%22%3A%2220:2B:C1:E0:06:65%22%7D'
 - Search for devices that have not initiated an inform in the last 7 days.
 .. code:: javascript
  query = {
    "_lastInform": {
      "$lt" : "2017-12-11 13:16:23 +0000"
    }
  }
 .. code:: bash
  curl -i 'http://localhost:7557/devices/?query=%7B%22_lastInform%22%3A%7B%22%24lt%22%3A%222017-12-11%2013%3A16%3A23%20%2B0000%22%7D%7D'
 - Show pending tasks for a given device:
 .. code:: javascript
  query = {"device": "202BC1-BM632w-000000"}
 .. code:: bash
  curl -i 'http://localhost:7557/tasks/?query=%7B%22device%22%3A%22202BC1-BM632w-000000%22%7D'
 - Return specific parameters for a given device:
 .. code:: javascript
  query = {"_id": "202BC1-BM632w-000000"}
 .. code:: bash
  curl -i 'http://localhost:7557/devices?query=%7B%22_id%22%3A%22202BC1-BM632w-000000%22%7D&projection=InternetGatewayDevice.DeviceInfo.ModelName,InternetGatewayDevice.DeviceInfo.Manufacturer'
 The ``projection`` URL param is a comma-separated list of the parameters to receive.
 POST /devices/\<device_id\>/tasks?[connection_request]
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Enqueue task(s) and optionally trigger a connection request to the device.
 Refer to :ref:`tasks` section for information about the task object format.
 Returns status code 200 if the tasks have been successfully executed, and 202
 if the tasks have been queued to be executed at the next inform.
 *device_id*: The ID of the device.
 *connection_request*: Indicates that a connection request will be triggered to
 execute the tasks immediately. Otherwise, the tasks will be queued and be
 processed at the next inform.
 The response body is the task object as it is inserted in the database. The
 object will include ``_id`` property which you can use to look up the task
 later.
 Examples
 ^^^^^^^^
 - Refresh all device parameters now:
 .. code:: bash
  curl -i 'http://localhost:7557/devices/202BC1-BM632w-000000/tasks?connection_request' \
  -X POST \
  --data '{"name": "refreshObject", "objectName": ""}'
 - Change WiFi SSID and password:
 .. code:: javascript
  {
    "name": "setParameterValues",
    "parameterValues": [
      ["InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.SSID", "GenieACS", "xsd:string"],
      ["InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.PreSharedKey.1.PreSharedKey", "hello world", "xsd:string"]
    ]
  }
 .. code:: bash
  curl -i 'http://localhost:7557/devices/202BC1-BM632w-000000/tasks?connection_request' \
  -X POST \
  --data '{"name":"setParameterValues", "parameterValues": [["InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.SSID", "GenieACS", "xsd:string"],["InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.PreSharedKey.1.PreSharedKey", "hello world", "xsd:string"]]}'
 POST /tasks/\<task_id\>/retry
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Retry a faulty task at the next inform.
 *task_id*: The ID of the task as returned by 'GET /tasks' request.
 Example
 ^^^^^^^
 .. code:: bash
  curl -i 'http://localhost:7557/tasks/5403908ef28ea3a25c138adc/retry' -X POST
 DELETE /tasks/\<task_id\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 Delete the given task.
 *task_id*: The ID of the task as returned by 'GET /tasks' request.
 Example
 ^^^^^^^
 .. code:: bash
  curl -i 'http://localhost:7557/tasks/5403908ef28ea3a25c138adc' -X DELETE
 DELETE /faults/\<fault_id\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Delete the given fault.
 *fault_id*: The ID of the fault as returned by 'GET /faults' request. The ID
 format is "\<device_id\>:\<channel\>".
 Example
 ^^^^^^^
 .. code:: bash
  curl -i 'http://localhost:7557/faults/202BC1-BM632w-000000:default' -X DELETE
 DELETE /devices/\<device_id\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Delete the given device from the database.
 Example
 ^^^^^^^
 .. code:: bash
  curl -X DELETE -i 'http://localhost:7557/devices/202BC1-BM632w-000001'
 .. note::
  Note that the device will be registered again when/if it contacts the ACS
  again (e.g. on the next periodic inform).
 POST /devices/\<device_id\>/tags/\<tag\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Assign a tag to a device. Has no effect if such tag already exists.
 *device_id*: The ID of the device.
 *tag*: The tag to be assigned.
 Example
 ^^^^^^^
 Assign the tag "testing" to a device:
 .. code:: bash
  curl -i 'http://localhost:7557/devices/202BC1-BM632w-000000/tags/testing' -X POST
 DELETE /devices/\<device_id\>/tags/\<tag\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Remove a tag from a device.
 *device_id*: The ID of the device.
 *tag*: The tag to be removed.
 Example
 ^^^^^^^
 Remove the tag "testing" from a device:
 .. code:: bash
  curl -i 'http://localhost:7557/devices/202BC1-BM632w-000000/tags/testing' -X DELETE
 PUT /presets/\<preset_name\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Create or update a preset. Returns status code 200 if the preset has been
 added/updated successfully. The body of the request is a JSON representation of
 the preset. Refer to :ref:`presets` section below for details about its format.
 *preset_name*: The name of the preset.
 Example
 ^^^^^^^
 Create a preset to set 5 minutes inform interval for all devices tagged with
 "test":
 .. code:: javascript
  query = {
    "weight": 0,
    "precondition": "{\"_tags\": \"test\"}"
    "configurations": [
      {
        "type": "value",
        "name": "InternetGatewayDevice.ManagementServer.PeriodicInformEnable",
        "value": "true"
      },
      {
        "type": "value",
        "name": "InternetGatewayDevice.ManagementServer.PeriodicInformInterval",
        "value": "300"
      }
    ]
  }
 .. code:: bash
  curl -i 'http://localhost:7557/presets/inform' \
  -X PUT \
  --data '{"weight": 0, "precondition": "{\"_tags\": \"test\"}", "configurations": [{"type": "value", "name": "InternetGatewayDevice.ManagementServer.PeriodicInformEnable", "value": "true"}, {"type": "value", "name": "InternetGatewayDevice.ManagementServer.PeriodicInformInterval", "value": "300"}]}'
 DELETE /presets/\<preset_name\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. code:: bash
 	curl -i 'http://localhost:7557/presets/inform' -X DELETE
 PUT /files/\<file_name\>
 ~~~~~~~~~~~~~~~~~~~~~~~~
 Upload a new file or overwrite an existing one. Returns status code 200 if the
 file has been added/updated successfully. The file content should be sent as
 the request body.
 *file_name*: The name of the uploaded file.
 The following file metadata may be sent as request headers:
 - ``fileType``: For firmware images it should be "1 Firmware Upgrade Image".
  Other common types are "2 Web Content" and "3 Vendor Configuration File".
 - ``oui``: The OUI of the device model that this file belongs to.
 - ``productClass``: The product class of the device.
 - ``version``: In case of firmware images, this refer to the firmware version.
 Example
 ^^^^^^^
 Upload a firmware image file:
 .. code:: bash
  curl -i 'http://localhost:7557/files/new_firmware_v1.0.bin' \
  -X PUT \
  --data-binary @"./new_firmware_v1.0.bin" \
  --header "fileType: 1 Firmware Upgrade Image" \
  --header "oui: 123456" \
  --header "productClass: ABC" \
  --header "version: 1.0"
 DELETE /files/\<file_name\>
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Delete a previously uploaded file:
 .. code:: bash
 	curl -i 'http://localhost:7557/files/new_firmware_v1.0.bin' -X DELETE
 GET /files/
 ~~~~~~~~~~~
 Gets all previously uploaded files.
 GET /files/?query={"filename":"\<filename\>"}
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Find files using a query.
 .. _tasks:
 Tasks
 -----
 Find the different available tasks and their object structure.
 ``getParameterValues``
 ~~~~~~~~~~~~~~~~~~~~~~
 .. code:: javascript
  query = {
    "name": "getParameterValues",
    "parameterNames": [
      "InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnectionNumberOfEntries",
      "InternetGatewayDevice.Time.NTPServer1", "InternetGatewayDevice.Time.Status"
    ]
  }
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-96318REF-SR360NA0A4%252D0003196/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "getParameterValues", "parameterNames": ["InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnectionNumberOfEntries", "InternetGatewayDevice.Time.NTPServer1", "InternetGatewayDevice.Time.Status"] }'
 You may request a single or multiple parameters at once.
 After the task has been executed successfully you can then fetch the CPE object
 and read the parameters from the JSON object.
 .. code:: javascript
  query = {"_id": "00236a-96318REF-SR360NA0A4%2D0003196"}
 .. code:: bash
  curl -i 'http://localhost:7557/devices/?query=%7B%22_id%22%3A%2200236a-96318REF-SR360NA0A4%252D0003196%22%7D'
 ``refreshObject``
 ~~~~~~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "refreshObject", "objectName": "InternetGatewayDevice.WANDevice.1.WANConnectionDevice"}'
 ``setParameterValues``
 ~~~~~~~~~~~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "setParameterValues", "parameterValues": [["InternetGatewayDevice.ManagementServer.UpgradesManaged",false]]}'
 Multiple values can be set at once by adding multiple arrays to the
 parameterValues key. For example:
 .. code:: javascript
  {
    name: "setParameterValues",
    parameterValues: [["InternetGatewayDevice.ManagementServer.UpgradesManaged", false], ["InternetGatewayDevice.Time.Enable", true], ["InternetGatewayDevice.Time.NTPServer1", "pool.ntp.org"]]
  }
 ``addObject``
 ~~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name":"addObject","objectName":"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection"}'
 ``deleteObject``
 ~~~~~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name":"deleteObject","objectName":"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.1"}'
 ``reboot``
 ~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "reboot"}'
 ``factoryReset``
 ~~~~~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "factoryReset"}'
 ``download``
 ~~~~~~~~~~~~
 .. code:: bash
  curl -i 'http://localhost:7557/devices/00236a-SR552n-SR552NA084%252D0003269/tasks?timeout=3000&connection_request' \
  -X POST \
  --data '{"name": "download", "file": "mipsbe-6-42-lite.xml"}'
 .. _presets:
 Presets
 -------
 Presets assign a set of configuration or a Provision script to devices based on
 a precondition (search filter), schedule (cron expression), and events.
 Precondition
 ~~~~~~~~~~~~
 The ``precondition`` property is a JSON string representation of the search
 filter to test if the preset applies to a given device. Examples preconditions
 are:
 - ``{"param": "value"}``
 - ``{"param": value", "param2": {"$ne": "value2"}}``
 Other operators that can be used are ``$gt``, ``$lt``, ``$gte`` and ``$lte``.
 Configuration
 ~~~~~~~~~~~~~
 The configuration property is an array containing the different configurations
 to be applied to a device, as shown below:
 .. code:: javascript
  [
    {
      "type": "value",
      "name": "InternetGatewayDevice.ManagementServer.PeriodicInformEnable",
      "value": "true"
    },
    {
      "type": "value",
      "name": "InternetGatewayDevice.ManagementServer.PeriodicInformInterval",
      "value": "300"
    },
    {
      "type": "delete_object",
      "name": "object_parent",
      "object": "object_name"
    },
    {
      "type": "add_object",
      "name": "object_parent",
      "object": "object_name"
    },
    {
      "type": "provision",
      "name": "YourProvisionName"
    },
  ] 
 The configuration type ``provision`` triggers a Provision script. In the
 example above, the provision named "YourProvisionName" will be executed.
 Provisions
 ----------
 Create a provision
 ~~~~~~~~~~~~~~~~~~
 The Provision's JavaScript code is the body of the HTTP PUT request.
 .. code:: bash
  curl -X PUT -i 'http://localhost:7557/provisions/mynewprovision' --data 'log("Provision started at " + now);'
 Delete a provision
 ~~~~~~~~~~~~~~~~~~
 .. code:: bash
  curl -X DELETE -i 'http://localhost:7557/provisions/mynewprovision'
 Get provisions
 ~~~~~~~~~~~~~~
 Get all provisions:
 .. code:: bash
  curl -X GET -i 'http://localhost:7557/provisions/'
@@ -1,65 +0,0 @@
 # Configuration file for the Sphinx documentation builder.
 #
 # This file only contains a selection of the most common options. For a full
 # list see the documentation:
 # http://www.sphinx-doc.org/en/master/config
 # -- Path setup --------------------------------------------------------------
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 # import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
 import json
 # -- Project information -----------------------------------------------------
 project = 'GenieACS Documentation'
 copyright = '2024, GenieACS Inc.'
 author = 'GenieACS Inc.'
 # The full version, including alpha/beta/rc tags
 release = json.load(open("../package.json"))["version"]
 # -- General configuration ---------------------------------------------------
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
 ]
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
 # -- Options for HTML output -------------------------------------------------
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
 # html_theme = 'alabaster'
 html_theme = 'sphinx_rtd_theme'
 html_logo = "logo.svg"
 html_theme_options = {
  "logo_only": True
 }
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 # html_static_path = ['_static']
 master_doc = 'index'
 highlight_language = 'javascript'
@@ -1,72 +0,0 @@
 .. _cpe-authentication:
 CPE Authentication
 ==================
 CPE to ACS
 ----------
 .. note::
  By default GenieACS will accept any incoming connection via HTTP/HTTPS and
  respond to it.
 The following parameters are used to set and get (password is redacted but
 can be set) the username/password used to authenticate against the ACS:
 Username: ``Device.ManagementServer.Username`` or ``InternetGatewayDevice.ManagementServer.Username``
 Password: ``Device.ManagementServer.Password`` or ``InternetGatewayDevice.ManagementServer.Password``
 Enable CPE to ACS Authentication
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 CPE to ACS authentication can be configured in the web interface by using the
 `Config` option in the `Admin` tab.
 Go to the `Admin` -> `Config` page and click on `New config` button at the
 bottom of the page. This will open pop-up which requires you to fill in a key
 and value. The key should be ``cwmp.auth``. The value accepts a boolean.
 Setting the value to ``true`` makes it so that GenieACS accepts any incoming
 connection, setting it to ``false`` makes GenieACS deny all incoming
 connections. This can be further configured using the ``AUTH()`` and ``EXT()``
 functions.
 The ``AUTH()`` function
 ~~~~~~~~~~~~~~~~~~~~~~~
 The ``AUTH()`` function accepts two parameters, username and password. It
 checks the given username and password with the incoming request to determine
 whether to return true or false.
 Basic usage of the ``AUTH()`` function could be as follows:
 .. code:: sql
   AUTH("fixed-username", "fixed-password")
 This will only accept incoming request who authenticate with
 "fixed-username" and "fixed-password".
 The various device parameters can be referenced from within the ``cwmp.auth``
 expression. For example:
 .. code:: sql
   AUTH(Device.ManagementServer.Username, Device.ManagementServer.Password)
 The ``EXT()`` function
 ~~~~~~~~~~~~~~~~~~~~~~
 The ``EXT()`` function makes it possible to call an :ref:`extension
 <extensions>` script from the auth expression. This can be used to fetch
 the credentials from an external source:
 .. code:: sql
   AUTH(DeviceID.SerialNumber, EXT("authenticate", "getPassword", DeviceID.SerialNumber))
 ACS to CPE
 ----------
 TODO
@@ -1,217 +0,0 @@
 .. _environment-variables:
 Environment Variables
 =====================
 Configuring GenieACS services can be done through the following environment
 variables:
 .. attention::
  All GenieACS environment variables must be prefixed with ``GENIEACS_``.
 MONGODB_CONNECTION_URL
  MongoDB connection string.
  Default: ``mongodb://127.0.0.1/genieacs``
 EXT_DIR
  The directory from which to look up extension scripts.
  Default: ``<installation dir>/config/ext``
 EXT_TIMEOUT
  Timeout (in milliseconds) to allow for calls to extensions to return a
  response.
  Default: ``3000``
 DEBUG_FILE
  File to dump CPE debug log.
  Default: unset
 DEBUG_FORMAT
  Debug log format. Valid values are 'yaml' and 'json'.
  Default: ``yaml``
 LOG_FORMAT
  The format used for the log entries in ``CWMP_LOG_FILE``, ``NBI_LOG_FILE``,
  ``FS_LOG_FILE``, and ``UI_LOG_FILE``. Possible values are ``simple`` and
  ``json``.
  Default: ``simple``
 ACCESS_LOG_FORMAT
  The format used for the log entries in ``CWMP_ACCESS_LOG_FILE``,
  ``NBI_ACCESS_LOG_FILE``, ``FS_ACCESS_LOG_FILE``, and ``UI_ACCESS_LOG_FILE``.
  Possible values are ``simple`` and ``json``.
  Default: ``simple``
 CWMP_WORKER_PROCESSES
  The number of worker processes to spawn for genieacs-cwmp. A value of 0 means
  as many as there are CPU cores available.
  Default: ``0``
 CWMP_PORT
  The TCP port that genieacs-cwmp listens on.
  Default: ``7547``
 CWMP_INTERFACE
  The network interface that genieacs-cwmp binds to.
  Default: ``::``
 CWMP_SSL_CERT
  Path to certificate file. If omitted, non-secure HTTP will be used.
  Default: unset
 CWMP_SSL_KEY
  Path to certificate key file. If omitted, non-secure HTTP will be used.
  Default: unset
 CWMP_LOG_FILE
  File to log process related events for genieacs-cwmp. If omitted, logs will
  go to stderr.
  Default: unset
 CWMP_ACCESS_LOG_FILE
  File to log incoming requests for genieacs-cwmp. If omitted, logs will go to
  stdout.
  Default: unset
 NBI_WORKER_PROCESSES
  The number of worker processes to spawn for genieacs-nbi. A value of 0 means
  as many as there are CPU cores available.
  Default: ``0``
 NBI_PORT
  The TCP port that genieacs-nbi listens on.
  Default: ``7557``
 NBI_INTERFACE
  The network interface that genieacs-nbi binds to.
  Default: ``::``
 NBI_SSL_CERT
  Path to certificate file. If omitted, non-secure HTTP will be used.
  Default: unset
 NBI_SSL_KEY
  Path to certificate key file. If omitted, non-secure HTTP will be used.
  Default: unset
 NBI_LOG_FILE
  File to log process related events for genieacs-nbi. If omitted, logs will go
  to stderr.
  Default: unset
 NBI_ACCESS_LOG_FILE
  File to log incoming requests for genieacs-nbi. If omitted, logs will go to
  stdout.
  Default: unset
 FS_WORKER_PROCESSES
  The number of worker processes to spawn for genieacs-fs. A value of 0 means
  as many as there are CPU cores available.
  Default: ``0``
 FS_PORT
  The TCP port that genieacs-fs listens on.
  Default: ``7567``
 FS_INTERFACE
  The network interface that genieacs-fs binds to.
  Default: ``::``
 FS_SSL_CERT
  Path to certificate file. If omitted, non-secure HTTP will be used.
  Default: unset
 FS_SSL_KEY
  Path to certificate key file. If omitted, non-secure HTTP will be used.
  Default: unset
 FS_LOG_FILE
  File to log process related events for genieacs-fs. If omitted, logs will go
  to stderr.
  Default: unset
 FS_ACCESS_LOG_FILE
  File to log incoming requests for genieacs-fs. If omitted, logs will go to
  stdout.
  Default: unset
 FS_URL_PREFIX
  The URL prefix (e.g. 'https://example.com:7567/') to use when generating the
  file URL for TR-069 Download requests. Set this if genieacs-fs and
  genieacs-cwmp are behind a proxy or running on different servers.
  Default: auto generated based on the hostname from the ACS URL, FS_PORT
  config, and whether or not SSL is enabled for genieacs-fs.
 UI_WORKER_PROCESSES
  The number of worker processes to spawn for genieacs-ui. A value of 0 means
  as many as there are CPU cores available.
  Default: ``0``
 UI_PORT
  The TCP port that genieacs-ui listens on.
  Default: ``3000``
 UI_INTERFACE
  The network interface that genieacs-ui binds to.
  Default: ``::``
 UI_SSL_CERT
  Path to certificate file. If omitted, non-secure HTTP will be used.
  Default: unset
 UI_SSL_KEY
  Path to certificate key file. If omitted, non-secure HTTP will be used.
  Default: unset
 UI_LOG_FILE
  File to log process related events for genieacs-ui. If omitted, logs will go
  to stderr.
  Default: unset
 UI_ACCESS_LOG_FILE
  File to log incoming requests for genieacs-ui. If omitted, logs will go to
  stdout.
  Default: unset
 UI_JWT_SECRET
  The key used for signing JWT tokens that are stored in browser cookies. The
  string can be up to 64 characters in length.
  Default: unset
@@ -1,37 +0,0 @@
 // This is an example GenieACS extension to get the current latitude/longitude
 // of the International Space Station. Why, you ask? Because why not.
 // To install, copy this file to config/ext/iss.js.
 "use strict";
 const http = require("http");
 let cache = null;
 let cacheExpire = 0;
 function latlong(args, callback) {
  if (Date.now() < cacheExpire) return callback(null, cache);
  http
    .get("http://api.open-notify.org/iss-now.json", (res) => {
      if (res.statusCode !== 200)
        return callback(
          new Error(`Request failed (status code: ${res.statusCode})`),
        );
      let rawData = "";
      res.on("data", (chunk) => (rawData += chunk));
      res.on("end", () => {
        let pos = JSON.parse(rawData)["iss_position"];
        cache = [+pos["latitude"], +pos["longitude"]];
        cacheExpire = Date.now() + 10000;
        callback(null, cache);
      });
    })
    .on("error", (err) => {
      callback(err);
    });
 }
 exports.latlong = latlong;
@@ -1,33 +0,0 @@
 .. _extensions:
 Extensions
 ==========
 Given that :ref:`provisions` and :ref:`virtual-parameters` are executed in a
 sandbox environment, it is not possible to interact with external sources or
 execute any action that requires OS, file system, or network access. Extensions
 exist to bridge that gap.
 Extensions are fully-privileged Node.js modules and as such have access to
 standard Node libraries and 3rd party packages. Functions exposed by the
 extension can be called from Provision scripts using the ``ext()`` function. A
 typical use case for extensions is fetching credentials from a database to have
 that pushed to the device during provisioning.
 By default, the extension JS code must be placed under ``config/ext``
 directory. You may need to create that directory if it doesn't already exist.
 The example extension below fetches data from an external REST API and returns
 that to the caller:
 .. literalinclude:: ext-sample.js
  :language: javascript
 To call this extension from a Provision or a Virtual Parameter script:
 .. code:: javascript
  // The arguments "arg1" and "arg2" are passed to the latlong. Though they are
  // unused in this particular example.
  const res = ext("ext-sample", "latlong", "arg1", "arg2");
  log(JSON.stringify(res));
@@ -1,6 +0,0 @@
 .. _https:
 HTTPS
 =====
 TODO
@@ -1,56 +0,0 @@
 .. GenieACS documentation master file, created by
   sphinx-quickstart on Wed Jun  5 13:47:06 2019.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.
 Welcome to GenieACS's documentation!
 ====================================
 .. toctree::
  :caption: Table of Contents
 .. raw:: latex
   \part{Installation}
 .. toctree::
  :maxdepth: 1
  :caption: Installation
  installation-guide
  environment-variables
 .. raw:: latex
   \part{Administration}
 .. toctree::
  :maxdepth: 1
  :caption: Administration
  provisions
  virtual-parameters
  administration-faq
 .. raw:: latex
   \part{Integration}
 .. toctree::
  :maxdepth: 1
  :caption: Integration
  extensions
  api-reference
 .. raw:: latex
   \part{Security}
 .. toctree::
  :maxdepth: 1
  :caption: Security
  https
  cpe-authentication
  roles-and-permissions
@@ -1,238 +0,0 @@
 Installation Guide
 ==================
 This guide is for installing GenieACS on a single server on any Linux distro
 that uses *systemd* as its init system.
 The various GenieACS services are independent of each other and may be
 installed on different servers. You may also run multiple instances of each in
 a load-balancing/failover setup.
 .. attention::
  For production deployments make sure to configure TLS and change
  ``UI_JWT_SECRET`` to a unique and secure string. Refer to :ref:`https`
  section for how to enable TLS to encrypt traffic.
 Prerequisites
 -------------
 .. topic:: Node.js
  GenieACS requires Node.js 12.13 and up. Refer to https://nodejs.org/ for
  instructions.
 .. topic:: MongoDB
  GenieACS requires MongoDB 3.6 and up. Refer to https://www.mongodb.com/ for
  instructions.
 Install GenieACS
 -------------------
 .. topic:: Installing from NPM:
  .. parsed-literal::
    sudo npm install -g genieacs@\ |release|
 .. topic:: Installing from source
  If you prefer installing from source, such as when running a GenieACS copy
  with custom patches, refer to README.md file in the source package. Adjust
  the next steps below accordingly.
 Configure systemd
 -----------------
 .. topic:: Create a system user to run GenieACS daemons
  .. code:: bash
    sudo useradd --system --no-create-home --user-group genieacs
 .. topic:: Create directory to save extensions and environment file
  We'll use :file:`/opt/genieacs/ext/` directory to store extension scripts (if any).
  .. code:: bash
    mkdir /opt/genieacs
    mkdir /opt/genieacs/ext
    chown genieacs:genieacs /opt/genieacs/ext
  Create the file :file:`/opt/genieacs/genieacs.env` to hold our configuration
  options which we pass to GenieACS as environment variables. See
  :ref:`environment-variables` section for a list of all available
  configuration options.
  .. code:: bash
    GENIEACS_CWMP_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-cwmp-access.log
    GENIEACS_NBI_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-nbi-access.log
    GENIEACS_FS_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-fs-access.log
    GENIEACS_UI_ACCESS_LOG_FILE=/var/log/genieacs/genieacs-ui-access.log
    GENIEACS_DEBUG_FILE=/var/log/genieacs/genieacs-debug.yaml
    NODE_OPTIONS=--enable-source-maps
    GENIEACS_EXT_DIR=/opt/genieacs/ext
  Generate a secure JWT secret and append to :file:`/opt/genieacs/genieacs.env`:
  .. code:: bash
    node -e "console.log(\"GENIEACS_UI_JWT_SECRET=\" + require('crypto').randomBytes(128).toString('hex'))" >> /opt/genieacs/genieacs.env
  Set file ownership and permissions:
  .. code:: bash
    sudo chown genieacs:genieacs /opt/genieacs/genieacs.env
    sudo chmod 600 /opt/genieacs/genieacs.env
 .. topic:: Create logs directory
  .. code:: bash
    mkdir /var/log/genieacs
    chown genieacs:genieacs /var/log/genieacs
 .. topic:: Create systemd unit files
  Create a systemd unit file for each of the four GenieACS services. Note that
  we're using EnvironmentFile directive to read the environment variables from
  the file we created earlier.
  Each service has two streams of logs: access log and process log. Access logs
  are configured here to be dumped in a log file under
  :file:`/var/log/genieacs/` while process logs go to *journald*. Use
  ``journalctl`` command to view process logs.
  .. attention::
    If the command :command:`systemctl edit --force --full` fails, you can
    create the unit file manually.
  1. Run the following command to create ``genieacs-cwmp`` service:
    .. code:: bash
      sudo systemctl edit --force --full genieacs-cwmp
    Then paste the following in the editor and save:
    .. code:: cfg
      [Unit]
      Description=GenieACS CWMP
      After=network.target
      [Service]
      User=genieacs
      EnvironmentFile=/opt/genieacs/genieacs.env
      ExecStart=/usr/bin/genieacs-cwmp
      [Install]
      WantedBy=default.target
  2. Run the following command to create ``genieacs-nbi`` service:
    .. code:: bash
      sudo systemctl edit --force --full genieacs-nbi
    Then paste the following in the editor and save:
    .. code:: cfg
      [Unit]
      Description=GenieACS NBI
      After=network.target
      [Service]
      User=genieacs
      EnvironmentFile=/opt/genieacs/genieacs.env
      ExecStart=/usr/bin/genieacs-nbi
      [Install]
      WantedBy=default.target
  3. Run the following command to create ``genieacs-fs`` service:
    .. code:: bash
      sudo systemctl edit --force --full genieacs-fs
    Then paste the following in the editor and save:
    .. code:: cfg
      [Unit]
      Description=GenieACS FS
      After=network.target
      [Service]
      User=genieacs
      EnvironmentFile=/opt/genieacs/genieacs.env
      ExecStart=/usr/bin/genieacs-fs
      [Install]
      WantedBy=default.target
  4. Run the following command to create ``genieacs-ui`` service:
    .. code:: bash
      sudo systemctl edit --force --full genieacs-ui
    Then paste the following in the editor and save:
    .. code:: cfg
      [Unit]
      Description=GenieACS UI
      After=network.target
      [Service]
      User=genieacs
      EnvironmentFile=/opt/genieacs/genieacs.env
      ExecStart=/usr/bin/genieacs-ui
      [Install]
      WantedBy=default.target
 .. topic:: Configure log file rotation using logrotate
  Save the following as :file:`/etc/logrotate.d/genieacs`
  .. code::
    /var/log/genieacs/*.log /var/log/genieacs/*.yaml {
        daily
        rotate 30
        compress
        delaycompress
        dateext
    }
 .. topic:: Enable and start services
  .. code:: bash
    sudo systemctl enable genieacs-cwmp
    sudo systemctl start genieacs-cwmp
    sudo systemctl status genieacs-cwmp
    sudo systemctl enable genieacs-nbi
    sudo systemctl start genieacs-nbi
    sudo systemctl status genieacs-nbi
    sudo systemctl enable genieacs-fs
    sudo systemctl start genieacs-fs
    sudo systemctl status genieacs-fs
    sudo systemctl enable genieacs-ui
    sudo systemctl start genieacs-ui
    sudo systemctl status genieacs-ui
  Review the status message for each to verify that the services are running
  successfully.
@@ -1 +0,0 @@
 <svg xmlns="http://www.w3.org/2000/svg" width="310" height="64" viewBox="0 0 82.021 16.933"><path d="M7.341 9.892l.687 1.307c-.305.53-.718.99-1.238 1.378-.851.503-1.543.421-2.076-.244 1.03-.466 1.907-1.28 2.627-2.44M4.676 5.97a2.144 2.144 0 0 1 .237 1.228 8.358 8.358 0 0 1-.518 2.21 7.386 7.386 0 0 0-.424 1.708 2.38 2.38 0 0 0 .168 1.205c.163.393.457.73.883 1.01-.458.138-.859.175-1.202.112-.6-.114-.992-.476-1.175-1.084a3.571 3.571 0 0 1-.093-1.787 11.742 11.742 0 0 1 .435-1.644 11.606 11.606 0 0 0 .432-1.708c.037-.22.041-.45.01-.687a1.66 1.66 0 0 0-.273-.706 1.582 1.582 0 0 0-.6-.506c.245-.046.487-.083.728-.113a1.45 1.45 0 0 1 .728.123c.295.13.517.343.665.639m7.901-1.802l.222.218-1.847 1.678.555.555 1.678-1.846.218.22-1.843 1.679.521.521L13.76 5.35l.222.218-1.847 1.678.556.556 1.764-1.765.375.368-1.584 1.584h.004l-2.065 2.061-1.253.515-1.795-1.348-.007-.008-.008.008.004.007 1.194 1.592-.398.161-1.75-1.75.444-1.08 1.636 1.228.004.003.011-.007-.003-.004-1.39-1.85.225-.553 1.694-1.689h-.004l1.956-1.952.372.368-1.769 1.768.552.552 1.678-1.843m1.456 9.91c1.55-1.549 2.324-3.42 2.324-5.612 0-2.187-.774-4.058-2.323-5.612C12.48 1.304 10.61.53 8.423.53 6.23.53 4.36 1.305 2.81 2.854c-.95.953-1.609 2.025-1.974 3.213l.158-.086c.788-.415 1.486-.203 2.094.638-.55.113-.992.232-1.325.357a1.923 1.923 0 0 0-.503.289c-.337.268-.55.608-.638 1.02a3.98 3.98 0 0 0 .011 1.761c.285 1.527 1.011 2.87 2.178 4.032 1.549 1.552 3.42 2.328 5.612 2.328 2.19 0 4.06-.776 5.612-2.328" fill="white"/><path d="M47.104 1.767c-.456 0-.816.388-.816.849 0 .46.355.816.816.816a.82.82 0 0 0 .833-.816.85.85 0 0 0-.833-.849zM25.602 3.171c-.205 0-.382.046-.5.13-.118.085-.187.201-.187.4v.981l-.151-.031a9.89 9.89 0 0 0-1.872-.173c-1.288 0-2.148.066-2.657.374a1.264 1.264 0 0 0-.548.676c-.119.313-.177.73-.177 1.28v6.123c0 .982.25 1.508.787 1.824.539.315 1.407.41 2.628.41 1.308 0 2.178-.13 2.701-.484.524-.354.745-.932.745-1.942 0-.821-.147-1.37-.543-1.722-.395-.352-1.075-.528-2.186-.528h-2.787V9.255l.148.03c.478.092 1.094.11 1.89.11 1.292 0 2.148-.112 2.658-.453.51-.34.724-.915.724-1.958V3.7c0-.198-.07-.314-.186-.399a.843.843 0 0 0-.487-.13zm6.112 1.29c-1.226 0-2.038.137-2.526.495-.488.357-.697.945-.697 1.947v2.84c0 .935.23 1.54.743 1.932.512.39 1.337.571 2.544.571 1.392 0 2.17-.058 2.567-.196.199-.069.298-.15.355-.246.058-.097.078-.227.078-.405 0-.208-.04-.378-.117-.485-.076-.108-.186-.171-.396-.171-.794 0-1.057.19-2.408.19-.593 0-1.06-.078-1.395-.255-.335-.177-.53-.473-.53-.84v-.89h4.301a.717.717 0 0 0 .539-.213c.13-.138.213-.342.213-.62V6.903c0-1.037-.179-1.623-.654-1.97-.476-.348-1.3-.472-2.617-.472zm8.83 0c-1.303 0-2.117.086-2.59.49-.472.403-.665 1.168-.665 2.623v4.002c0 .215.074.338.197.426.123.088.31.134.524.134.223 0 .412-.046.537-.134.125-.088.198-.211.198-.426v-3.81c0-.424.006-.754.04-1.019.036-.265.102-.47.236-.623.135-.152.328-.236.574-.28.246-.044.552-.053.949-.053.4 0 .712.01.963.053.25.044.447.126.587.278.139.151.211.358.25.623.038.265.046.597.046 1.02v3.81c0 .217.07.34.19.427.122.088.308.134.53.134.215 0 .401-.046.525-.134.123-.088.196-.211.196-.426V7.574c0-1.455-.201-2.219-.682-2.623-.48-.404-1.303-.49-2.605-.49zm12.996 0c-1.227 0-2.038.137-2.527.495-.488.357-.696.945-.696 1.947v2.84c0 .935.23 1.54.742 1.932.513.39 1.338.571 2.545.571 1.392 0 2.17-.058 2.567-.196.2-.069.298-.15.355-.246.058-.097.079-.227.079-.405 0-.208-.04-.378-.117-.485-.077-.108-.188-.171-.398-.171-.793 0-1.056.19-2.406.19-.594 0-1.062-.078-1.396-.255-.335-.177-.53-.473-.53-.84v-.89h4.301a.716.716 0 0 0 .539-.213c.13-.138.213-.342.213-.62V6.903c0-1.037-.18-1.623-.655-1.97-.475-.348-1.3-.472-2.616-.472zm-6.436.13c-.212 0-.398.047-.522.14-.124.09-.198.22-.198.435v6.41c0 .215.073.338.196.426.123.088.31.134.524.134.223 0 .414-.046.54-.134.124-.088.198-.211.198-.426v-6.41c0-.215-.075-.342-.201-.434-.126-.092-.316-.142-.537-.142zM22.892 5.726c.662 0 1.4.065 1.918.146l.105.016v.92c0 .246-.01.453-.058.632a.757.757 0 0 1-.292.441c-.305.216-.798.263-1.673.263h-.123v-.013c-.761 0-1.26-.002-1.55-.16a.637.637 0 0 1-.307-.395c-.049-.17-.057-.37-.057-.625v-.062c0-.227.03-.418.108-.58a.811.811 0 0 1 .373-.373c.337-.171.826-.21 1.556-.21zm8.839.049c.677 0 1.116.054 1.412.237a.775.775 0 0 1 .316.373c.063.154.085.33.085.535v.811h-3.611V6.92c0-.437.135-.755.439-.928.304-.174.739-.217 1.359-.217zm21.825 0c.677 0 1.117.054 1.413.237a.779.779 0 0 1 .318.373c.062.154.084.33.084.535v.811h-3.613V6.92c0-.437.136-.755.44-.928.304-.174.738-.217 1.358-.217zm-32.701 5.867h2.388c.613 0 1.05.045 1.355.226.306.18.443.505.443.95 0 .429-.179.743-.533.907-.354.164-.861.206-1.568.206-.652 0-1.151-.045-1.51-.186a.952.952 0 0 1-.425-.3.789.789 0 0 1-.15-.483v-1.195z" fill="white"/><path d="M62.339 4.462c-1.399 0-2.18.078-2.58.225-.2.073-.3.157-.357.254-.058.097-.079.22-.079.386 0 .199.043.362.122.469.079.107.19.172.392.172.38 0 .616-.048.947-.096a9.457 9.457 0 0 1 1.475-.098c.58 0 .966.043 1.233.192a.685.685 0 0 1 .292.312c.06.13.082.281.082.452v.857h-2.484c-1.134 0-1.763.19-2.11.559-.346.367-.443.95-.443 1.802 0Line truncated
@@ -1,279 +0,0 @@
 .. _provisions:
 Provisions
 ==========
 A Provision is a piece of JavaScript code that is executed on the server on a
 per-device basis. It enables implementing complex provisioning scenarios and
 other operations such as automated firmware upgrade rollout. Apart from a few
 special functions, the script is essentially a standard ES6 code executed in
 strict mode.
 Provisions are mapped to devices using presets. Note that the added performance
 overhead when using Provisions as opposed to simple preset configuration
 entries is relatively small. Anything that can be done via preset
 configurations can be done using a Provision script. In fact, the now
 deprecated configuration format is still supported primarily for backward
 compatibility and it is recommended to use Provision scripts for all
 configuration.
 When assigning a Provision script to a preset, you may pass arguments to the
 script. The arguments can be accessed from the script through the global
 ``args`` variable.
 .. note::
  Provision scripts may get executed multiple times in a given session.
  Although all data model-mutating operations are idempotent, a script as a
  whole may not be. It is, therefore, necessary to repeatedly run the script
  until there are no more side effects and a stable state is reached.
 Built-in functions
 ------------------
 ``declare(path, timestamps, values)``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 This function is for declaring parameter values to be set, as well as specify
 constraints on how recent you'd like the parameter value (or other attributes)
 to have been refreshed from the device. If the given timestamp is lower than
 the timestamp of the last refresh from the device, then this function will
 return the last known value. Otherwise, the value will be fetched from the
 device before being returned to the caller.
 The timestamp argument is an object where the key is the attribute name (e.g.
 ``value``, ``object``, ``writable``, ``path``) and the value is an integer
 representing a Unix timestamp.
 The values argument is an object similar to the timestamp argument but its
 property values being the parameter values to be set.
 The possible attributes in 'timestamps' and 'values' arguments are:
 - ``value``: a [<value>, <type>] pair
 This attribute is not available for objects or object instances. If the value
 is not a [<value>, <type>] array then it'll assumed to be a value without a
 type and therefore the type will be inferred from the parameter's type.
 - ``writable``: boolean
 The meaning of this attribute can vary depending on the type of the parameter.
 In the case of regular parameters, it indicates if its value is writable. In
 the case of objects, it's whether or not it's possible to add new object
 instances. In the case of object instances, it indicates whether or not this
 instance can be deleted.
 - ``object``: boolean
 True if this is an object or object instance, false otherwise.
 - ``path``: string
 This attribute is special in that it's not a parameter attribute per se, but it
 refers to the presence of parameters matching the given path. For example,
 given the following wildcard path:
 ``InternetGatewayDevice.LANDevice.1.Hosts.Host.*.MACAddress``
 Using a recent timestamp for path in ``declare()`` will result in a sync with
 the device to rediscover all Host instances (``Host.*``). The path attribute
 can also be used to create or delete object instances as described in
 :ref:`path-format` section.
 The return value of ``declare()`` is an iterator to access parameters that
 match the given path. Each item in the iterator has the attribute 'path' in
 addition to any other attribute given in the ``declare()`` call. The iterator
 object itself has convenience attribute accessors which come in handy when
 you're expecting a single parameter (e.g. when path does not contain wildcards
 or aliases).
 .. code:: javascript
  // Example: Setting the SSID as the last 6 characters of the serial number
  let serial = declare("Device.DeviceInfo.SerialNumber", {value: 1});
  declare("Device.LANDevice.1.WLANConfiguration.1.SSID", null, {value: serial.value[0]});
 ``clear(path, timestamp)``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 This function invalidates the database copy of parameters (and their child
 parameters) that match the given path and have a last refresh timestamp that is
 less than the given timestamp. The most obvious use for this function is to
 invalidate the database copy of the entire data model after the device has been
 factory reset:
 .. code:: javascript
  // Example: Clear cached device data model Note
  // Make sure to apply only on "0 BOOTSTRAP" event
  clear("Device", Date.now());
  clear("InternetGatewayDevice", Date.now());
 ``commit()``
 ~~~~~~~~~~~~
 This function commits the pending declarations and performs any necessary sync
 with the device. It's usually not required to call this function as it called
 implicitly at the end of the script and when accessing any property of the
 promise-like object returned by the ``declare()`` function. Calling this
 explicitly is only necessary if you want to control the order in which
 parameters are configured.
 ``ext(file, function, arg1, arg2, ...)``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Execute an extension script and return the result. The first argument is the
 script filename while second argument is the function name within that script.
 Any remaining arguments will be passed to that function. See :ref:`extensions`
 for more details.
 ``log(message)``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Prints out a string in genieacs-cwmp's access log. It's meant to be used for
 debugging. Note that you may see multiple log entries as the script can be
 executed multiple times in a session. See :ref:`this FAQ
 <administration-faq-duplicate-log-entries>`.
 .. _path-format:
 Path format
 -----------
 A parameter path may contain a wildcard (``*``) or an alias filter
 (``[name:value]``). A wildcard segment in a parameter path will apply the
 declared configuration to zero or more parameters that match the given path
 where the wildcard segment can be anything.
 An alias filter is like a wildcard, but additionally performs filtering on the
 child parameters based on the key-value pairs provided. For example, the
 following path:
 ``Device.WANDevice.1.WANConnectionDevice.1.WANIPConnection.[AddressingType:DHCP].ExternalIPAddress``
 will return a list of ExternalIPAddress parameters (0 or more) where the
 sibling parameter AddressingType is assigned the value "DHCP".
 This can be useful when the exact instance numbers may be different from one
 device to another. It is possible to use more than one key-value pair in the
 alias filter. It's also possible to use multiple filters or use a combination
 of filters and wildcards.
 Creating/deleting object instances
 ----------------------------------
 Given the declarative nature of provisions, we cannot explicitly tell the
 device to create or delete an instance under a given object. Instead, we
 specify the number of instances we want there to be, and based on that GenieACS
 will determine whether or not it needs to create or delete instances. For
 example, the following declaration will ensure we have one and only one
 WANIPConnection object:
 .. code:: javascript
  // Example: Ensure we have one and only one WANIPConnection object
  declare("InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.*", null, {path: 1});
 Note the wildcard at the end of the parameter path.
 It is also possible to use alias filters as the last path segment which will
 ensure that the declared number of instances is satisfied given the alias
 filter:
 .. code:: javascript
  // Ensure that *all* other instances are deleted
  declare("InternetGatewayDevice.X_BROADCOM_COM_IPAddrAccCtrl.X_BROADCOM_COM_IPAddrAccCtrlListCfg.[]", null, {path: 0});
  // Add the two entries we care about
  declare("InternetGatewayDevice.X_BROADCOM_COM_IPAddrAccCtrl.X_BROADCOM_COM_IPAddrAccCtrlListCfg.[SourceIPAddress:192.168.1.0,SourceNetMask:255.255.255.0]",  {path: now}, {path: 1});
  declare("InternetGatewayDevice.X_BROADCOM_COM_IPAddrAccCtrl.X_BROADCOM_COM_IPAddrAccCtrlListCfg.[SourceIPAddress:172.16.12.0,SourceNetMask:255.255.0.0]", {path: now}, {path: 1});
 Special GenieACS parameters
 ---------------------------
 In addition to the parameters exposed in the device's data model through
 TR-069, GenieACS has its own set of special parameters:
 ``DeviceID``
 ~~~~~~~~~~~~
 This parameter sub-tree includes the following read-only parameters:
 - ``DeviceID.ID``
 - ``DeviceID.SerialNumber``
 - ``DeviceID.ProductClass``
 - ``DeviceID.OUI``
 - ``DeviceID.Manufacturer``
 ``Tags``
 ~~~~~~~~
 The ``Tags`` root parameter is used to expose device tags in the data model.
 Tags appear as child parameters that are writable and have boolean value.
 Setting a tag to ``false`` will delete that tag, and setting the value of a
 non-existing tag parameter to ``true`` will create it.
 .. code:: javascript
  // Example: Remove "tag1", add "tag2", and read "tag3"
  declare("Tags.tag1", null, {value: false});
  declare("Tags.tag2", null, {value: true});
  let tag3 = declare("Tags.tag3", {value: 1});
 ``Reboot``
 ~~~~~~~~~~
 The ``Reboot`` root parameter hold the timestamp of the last reboot command.
 The parameter value is writable and declaring a timestamp value that is larger
 than the current value will trigger a reboot.
 .. code:: javascript
  // Example: Reboot the device only if it hasn't been rebooted in the past 300 seconds
  declare("Reboot", null, {value: Date.now() - (300 * 1000)});
 ``FactoryReset``
 ~~~~~~~~~~~~~~~~
 Works like ``Reboot`` parameter but for factory reset.
 .. code:: javascript
  // Example: Default the device to factory settings
  declare("FactoryReset", null, {value: Date.now()});
 ``Downloads``
 ~~~~~~~~~~~~~
 The ``Downloads`` sub-tree holds information about the last download
 command(s). A download command is represented as an instance (e.g.
 ``Downloads.1``) containing parameters such as ``Download`` (timestamp),
 ``LastFileType``, ``LastFileName``. The parameters ``FileType``, ``FileName``,
 ``TargetFileName`` and ``Download`` are writable and can be used to trigger a
 new download.
 .. code:: javascript
  declare("Downloads.[FileType:1 Firmware Upgrade Image]", {path: 1}, {path: 1});
  declare("Downloads.[FileType:1 Firmware Upgrade Image].FileName", {value: 1}, {value: "firmware-2017.01.tar"});
  declare("Downloads.[FileType:1 Firmware Upgrade Image].Download", {value: 1}, {value: Date.now()});
 Common file types are:
 - ``1 Firmware Upgrade Image``
 - ``2 Web Content``
 - ``3 Vendor Configuration File``
 - ``4 Tone File``
 - ``5 Ringer File``
 .. warning::
  Pushing a file to the device is often a service-interrupting operation. It's
  recommended to only trigger it on certain events such as ``1 BOOT`` or during
  a predetermined maintenance window).
 After the CPE had finished downloading and applying the config file, it will
 send a ``7 TRANSFER COMPLETE`` event. You may use that to trigger a reboot
 after the firmware image or configuration file had been applied.
@@ -1 +0,0 @@
 sphinx_rtd_theme==2.0.0
@@ -1,6 +0,0 @@
 .. _roles-and-permissions:
 Roles and Permissions
 =====================
 TODO
@@ -1,105 +0,0 @@
 .. _virtual-parameters:
 Virtual Parameters
 ==================
 Virtual parameters are user-defined parameters whose values are generated using
 a custom Javascript code. Virtual parameters behave just like regular
 parameters and appear in the data model under ``VirtualParameters.`` path.
 Virtual parameter names cannot contain a period (``.``).
 The execution environment for virtual parameters is almost identical to that of
 provisions. See :ref:`provisions` for more details and examples. The only
 differences between the scripts of provisions and virtual parameters are:
 - You can't pass custom arguments to virtual parameter scripts. Instead, the
  variable ``args`` will hold the current vparam timestamps and values as well
  as the declared timestamps and values. Like this:
 .. code:: javascript
  // [<declared attr timestamps, declared attr values>, <current attr timestamps>, <current attr values>]
  [{path: 1559849387191, value: 1559849387191}, {value: ["new val", "xsd:string"]}, {path: 1559840000000, value: 1559840000000}, {value: ["cur val", "xsd:string"]}]
 - Virtual parameter scripts must return an object containing the attributes of
  this parameter.
 .. note::
  Just like a regular parameter, creating a virtual parameter does not
  automatically add it to the parameter list for a device. It needs to fetched
  (manually or via a preset) before you can see it in the data model.
 Examples
 --------
 Unified MAC parameter across different device models
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. code:: javascript
  // Example: Unified MAC parameter across different device models
  let m = "00:00:00:00:00:00";
  let d = declare("Device.WANDevice.*.WANConnectionDevice.*.WANIPConnection.*.MACAddress", {value: Date.now()});
  let igd = declare("InternetGatewayDevice.WANDevice.*.WANConnectionDevice.*.WANPPPConnection.*.MACAddress", {value: Date.now()});
  if (d.size) {
    for (let p of d) {
      if (p.value[0]) {
        m = p.value[0];
        break;
      }
    }  
  }
  else if (igd.size) {
    for (let p of igd) {
      if (p.value[0]) {
        m = p.value[0];
        break;
      }
    }  
  }
  return {writable: false, value: [m, "xsd:string"]};
 Expose an external value as a virtual parameter
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. code:: javascript
  // Example: Expose an external value as a virtual parameter
  let serial = declare("DeviceID.SerialNumber", {value: 1});
  if (args[1].value) {
    ext("example-ext", "set", serial.value[0], args[1].value[0]);
    return {writable: true, value: [args[1].value[0], "xsd:string"]};
  }
  else {
    let v = ext("example-ext", "get", serial.value[0]);
    return {writable: true, value: [v, "xsd:string"]};
  }
 Create an editable virtual parameter for WPA passphrase
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. code:: javascript
  // Example: Create an editable virtual parameter for WPA passphrase
  let m = "";
  if (args[1].value) {
    m = args[1].value[0];
    declare("Device.WiFi.AccessPoint.1.Security.KeyPassphrase", null, {value: m});
    declare("InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.KeyPassphrase", null, {value: m});
  }
  else {
    let d = declare("Device.WiFi.AccessPoint.1.Security.KeyPassphrase", {value: Date.now()});
    let igd = declare("InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.KeyPassphrase", {value: Date.now()});
    if (d.size) {
      m = d.value[0];
    }
    else if (igd.size) {
      m = igd.value[0];  
    }
  }
  return {writable: true, value: [m, "xsd:string"]};
@@ -1,77 +0,0 @@
 import { defineConfig } from "@eslint/config-helpers";
 import eslint from "@eslint/js";
 import tseslint from "typescript-eslint";
 import eslintConfigPrettier from "eslint-config-prettier";
 import globals from "globals";
 export default defineConfig(
  {
    ignores: ["dist/", "test/*.js"],
  },
  eslint.configs.recommended,
  ...tseslint.configs.recommended,
  eslintConfigPrettier,
  {
    languageOptions: {
      globals: {
        ...globals.es2022,
        ...globals.node,
      },
      parserOptions: {
        projectService: true,
        tsconfigRootDir: import.meta.dirname,
      },
    },
    rules: {
      "@typescript-eslint/no-shadow": ["error", { allow: ["err"] }],
      "handle-callback-err": "error",
      "prefer-arrow-callback": "error",
      "no-buffer-constructor": "error",
      "prefer-const": ["error", { destructuring: "all" }],
      eqeqeq: ["error", "always", { null: "ignore" }],
      "@typescript-eslint/no-explicit-any": "off",
      "@typescript-eslint/no-use-before-define": [
        "error",
        { functions: false },
      ],
      "@typescript-eslint/explicit-function-return-type": [
        "error",
        { allowExpressions: true },
      ],
      "@typescript-eslint/no-floating-promises": "error",
      "@typescript-eslint/no-misused-promises": "error",
      "no-prototype-builtins": "off",
    },
  },
  {
    files: ["ui/**/*.ts", "ui/**/*.tsx"],
    languageOptions: {
      globals: {
        ...globals.browser,
      },
    },
  },
  {
    files: ["seed/*.js"],
    languageOptions: {
      parserOptions: {
        project: null,
      },
      globals: {
        declare: "readonly",
        clear: "readonly",
        commit: "readonly",
        ext: "readonly",
        log: "readonly",
        args: "readonly",
      },
    },
    rules: {
      "@typescript-eslint/explicit-function-return-type": "off",
      "@typescript-eslint/no-floating-promises": "off",
      "@typescript-eslint/no-misused-promises": "off",
      "@typescript-eslint/no-shadow": "off",
      "no-shadow": ["error", { allow: ["err", "total"] }],
    },
  },
 );
@@ -0,0 +1,257 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This file incorporates work covered by the following copyright and
 # permission notice:
 #
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 config = require './config'
 db = require './db'
 http = require 'http'
 common = require './common'
 util = require 'util'
 URL = require 'url'
 auth = require './auth'
 connectionRequest = (deviceId, callback) ->
  # ensure socket is reused in case of digest authentication
  agent = new http.Agent({maxSockets : 1})
  statusToError = (statusCode) ->
    switch statusCode
      when 200, 204
        null
      when 401
        new Error('Incorrect connection request credentials')
      when 0
        new Error('Device is offline')
      else
        new Error("Unexpected response code from device: #{statusCode}")
  conReq = (url, authString, callback) ->
    options = URL.parse(url)
    options.agent = agent
    if authString
      options.headers = {'Authorization' : authString}
    request = http.get(options, (res) ->
      if res.statusCode == 401 and res.headers['www-authenticate']?
        authHeader = auth.parseAuthHeader(res.headers['www-authenticate'])
      callback(res.statusCode, authHeader)
      # don't need body, go ahead and emit events to free up the socket for possible reuse
      res.resume()
    )
    request.on('error', (err) ->
      # error event when request is aborted
      request.abort()
      callback(0)
    )
    request.on('socket', (socket) ->
      socket.setTimeout(2000)
      socket.on('timeout', () ->
        request.abort()
      )
    )
  proj = {
    'Device.ManagementServer.ConnectionRequestURL._value' : 1,
    'InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value' : 1
  }
  db.devicesCollection.findOne({_id : deviceId}, proj, (err, device)->
    if err
      callback(err)
      return
    if device.Device? # TR-181 data model
      connectionRequestUrl = device.Device.ManagementServer.ConnectionRequestURL._value
    else # TR-098 data model
      connectionRequestUrl = device.InternetGatewayDevice.ManagementServer.ConnectionRequestURL._value
    # for testing
    #connectionRequestUrl = connectionRequestUrl.replace(/^(http:\/\/)([0-9\.]+)(\:[0-9]+\/[a-zA-Z0-9]+\/?$)/, '$110.1.1.254$3')
    conReq(connectionRequestUrl, null, (statusCode, authHeader) ->
      if statusCode == 401
        [username, password] = config.auth.connectionRequest(deviceId)
        if authHeader.method is 'Basic'
          authString = auth.basic(username, password)
        else if authHeader.method is 'Digest'
          uri = URL.parse(connectionRequestUrl)
          authString = auth.digest(username, password, uri.path, 'GET', null, authHeader)
        conReq(connectionRequestUrl, authString, (statusCode, authHeader) ->
          if statusCode == 0
            # Workaround for some devices unexpectedly closing the connection
            return conReq(connectionRequestUrl, authString, (statusCode) -> callback(statusToError(statusCode)))
          callback(statusToError(statusCode))
        )
      else
        callback(statusToError(statusCode))
    )
  )
 watchTask = (taskId, timeout, callback) ->
  setTimeout( () ->
    db.tasksCollection.findOne({_id : taskId}, {'_id' : 1, 'fault' : 1}, (err, task) ->
      return callback(err) if err
      if task
        timeout -= 500
        if task.fault?
          callback(null, 'fault')
        else if timeout <= 0
          callback(null, 'timeout')
        else
          watchTask(taskId, timeout, callback)
      else
        callback(null, 'completed')
    )
  , 500)
 expandParam = (param, aliases) ->
  params = [param]
  for a,aa of aliases
    if a == param or common.startsWith(a, "#{param}.")
      for p in aa
        params.push(p) if p[p.lastIndexOf('.') + 1] != '_'
  return params
 sanitizeTask = (task, aliases, callback) ->
  task.timestamp = new Date(task.timestamp ? Date.now())
  if task.expiry?
    if common.typeOf(task.expiry) is common.DATE_TYPE or isNaN(task.expiry)
      task.expiry = new Date(task.expiry)
    else
      task.expiry = new Date(task.timestamp.getTime() + +task.expiry * 1000)
  switch task.name
    when 'getParameterValues'
      projection = {}
      for p in task.parameterNames
        for pp in expandParam(p, aliases)
          projection[pp] = 1
      db.devicesCollection.findOne({_id : task.device}, projection, (err, device) ->
        parameterNames = []
        for k of projection
          if common.getParamValueFromPath(device, k)?
            parameterNames.push(k)
        task.parameterNames = parameterNames
        callback(task)
      )
    when 'setParameterValues'
      projection = {}
      values = {}
      for p in task.parameterValues
        for pp in expandParam(p[0], aliases)
          projection[pp] = 1
          values[pp] = p[1]
      db.devicesCollection.findOne({_id : task.device}, projection, (err, device) ->
        parameterValues = []
        for k of projection
          param = common.getParamValueFromPath(device, k)
          if param?
            parameterValues.push([k, values[k], param._type])
        task.parameterValues = parameterValues
        callback(task)
      )
    else
      # TODO implement setParameterValues
      callback(task)
 addAliases = (device, aliases) ->
  for k,v of aliases
    for p in v
      pp = p.split('.')
      obj = device
      for i in pp
        if not obj[i]?
          obj = null
          break
        obj = obj[i]
      device[k] = obj if obj?
 insertTasks = (tasks, aliases, callback) ->
  if tasks? and common.typeOf(tasks) isnt common.ARRAY_TYPE
    tasks = [tasks]
  else if not tasks? or tasks.length == 0
    return callback(tasks)
  counter = tasks.length
  for task in tasks
    sanitizeTask(task, aliases, (t) ->
      if t.uniqueKey?
        db.tasksCollection.remove({device : t.device, uniqueKey : t.uniqueKey}, (err, removed) ->
        )
      --counter
      if counter == 0
        db.tasksCollection.insert(tasks, (err, _tasks) ->
          #util.log("#{_task.device}: Added task #{_task.name}(#{_task._id})") for _task in _tasks
          callback(err, _tasks)
        )
    )
 deleteDevice = (deviceId, callback) ->
  db.tasksCollection.remove({'device' : deviceId}, (err, removed) ->
    return callback(err) if err
    db.devicesCollection.remove({'_id' : deviceId}, (err, removed) ->
      return callback(err) if err
      db.redisClient.del("#{deviceId}_presets_hash", "#{deviceId}_inform_hash", (err) ->
        callback(err)
      )
    )
  )
 exports.addAliases = addAliases
 exports.sanitizeTask = sanitizeTask
 exports.connectionRequest = connectionRequest
 exports.watchTask = watchTask
 exports.insertTasks = insertTasks
 exports.deleteDevice = deleteDevice
@@ -1,490 +0,0 @@
 import { ObjectId } from "mongodb";
 import { collections } from "./db/db.ts";
 import {
  deleteConfig,
  deleteFault as dbDeleteFault,
  deleteFile,
  deletePermission,
  deletePreset,
  deleteProvision,
  deleteTask,
  deleteUser,
  deleteVirtualParameter,
  putConfig,
  putPermission,
  putPreset,
  putProvision,
  putUser,
  putVirtualParameter,
 } from "./ui/db.ts";
 import * as common from "./util.ts";
 import * as cache from "./cache.ts";
 import { acquireLock, getToken, releaseLock } from "./lock.ts";
 import {
  getRevision,
  getConfig,
  getConfigExpression,
  getUsers,
 } from "./ui/local-cache.ts";
 import {
  httpConnectionRequest,
  udpConnectionRequest,
  xmppConnectionRequest,
 } from "./connection-request.ts";
 import { Task } from "./types.ts";
 import Expression, { Value } from "./common/expression.ts";
 import { hashPassword } from "./auth.ts";
 import { flattenDevice } from "./ui/db.ts";
 import { ResourceLockedError } from "./common/errors.ts";
 import * as config from "../lib/config.ts";
 const XMPP_CONFIGURED = !!config.get("XMPP_JID");
 export async function connectionRequest(
  deviceId: string,
  device?: Record<string, Value>,
 ): Promise<string> {
  if (!device) {
    const res = await collections.devices.findOne({ _id: deviceId });
    if (!res) throw new Error("No such device");
    device = flattenDevice(res);
  }
  let connectionRequestUrl,
    udpConnectionRequestAddress,
    stunEnable,
    connReqJabberId,
    username,
    password;
  if (device["InternetGatewayDevice.ManagementServer.ConnectionRequestURL"]) {
    connectionRequestUrl =
      device["InternetGatewayDevice.ManagementServer.ConnectionRequestURL"] ||
      "";
    udpConnectionRequestAddress =
      device[
        "InternetGatewayDevice.ManagementServer.UDPConnectionRequestAddress"
      ] || "";
    stunEnable =
      device["InternetGatewayDevice.ManagementServer.STUNEnable"] || "";
    connReqJabberId =
      device["InternetGatewayDevice.ManagementServer.ConnReqJabberID"] || "";
    username =
      device[
        "InternetGatewayDevice.ManagementServer.ConnectionRequestUsername"
      ] || "";
    password =
      device[
        "InternetGatewayDevice.ManagementServer.ConnectionRequestPassword"
      ] || "";
  } else {
    connectionRequestUrl =
      device["Device.ManagementServer.ConnectionRequestURL"] || "";
    udpConnectionRequestAddress =
      device["Device.ManagementServer.UDPConnectionRequestAddress"] || "";
    stunEnable = device["Device.ManagementServer.STUNEnable"] || "";
    connReqJabberId = device["Device.ManagementServer.ConnReqJabberID"] || "";
    username =
      device["Device.ManagementServer.ConnectionRequestUsername"] || "";
    password =
      device["Device.ManagementServer.ConnectionRequestPassword"] || "";
  }
  let remoteAddress;
  try {
    remoteAddress = new URL(connectionRequestUrl).hostname;
  } catch {
    return "Invalid connection request URL";
  }
  const snapshot = await getRevision();
  const now = Date.now();
  const evalCallback = (exp: Expression): Expression => {
    if (exp instanceof Expression.Parameter) {
      let name = exp.path.toString();
      if (name === "id") name = "DeviceID.ID";
      else if (name === "serialNumber") name = "DeviceID.SerialNumber";
      else if (name === "productClass") name = "DeviceID.ProductClass";
      else if (name === "oui") name = "DeviceID.OUI";
      else if (name === "remoteAddress")
        return new Expression.Literal(remoteAddress);
      else if (name === "username") return new Expression.Literal(username);
      else if (name === "password") return new Expression.Literal(password);
      return new Expression.Literal(device[name] ?? null);
    } else if (exp instanceof Expression.FunctionCall) {
      if (exp.name === "NOW") return new Expression.Literal(now);
      else if (exp.name === "REMOTE_ADDRESS")
        return new Expression.Literal(remoteAddress);
      else if (exp.name === "USERNAME") return new Expression.Literal(username);
      else if (exp.name === "PASSWORD") return new Expression.Literal(password);
    }
    return exp;
  };
  const configCallback = (exp: Expression): Expression.Literal => {
    const e = evalCallback(exp);
    if (e instanceof Expression.Literal) return e;
    return new Expression.Literal(null);
  };
  const UDP_CONNECTION_REQUEST_PORT = getConfig(
    snapshot,
    "cwmp.udpConnectionRequestPort",
    0,
    configCallback,
  );
  const CONNECTION_REQUEST_TIMEOUT = getConfig(
    snapshot,
    "cwmp.connectionRequestTimeout",
    2000,
    configCallback,
  );
  const CONNECTION_REQUEST_ALLOW_BASIC_AUTH = getConfig(
    snapshot,
    "cwmp.connectionRequestAllowBasicAuth",
    false,
    configCallback,
  );
  let authExp: Expression = getConfigExpression(
    snapshot,
    "cwmp.connectionRequestAuth",
  );
  if (!authExp) {
    authExp = new Expression.FunctionCall("AUTH", [
      new Expression.FunctionCall("USERNAME", []),
      new Expression.FunctionCall("PASSWORD", []),
    ]);
  }
  authExp = authExp.evaluate(evalCallback);
  const debug = getConfig(snapshot, "cwmp.debug", false, configCallback);
  let udpProm = Promise.resolve(false);
  if (udpConnectionRequestAddress && +stunEnable) {
    try {
      const u = new URL("udp://" + udpConnectionRequestAddress);
      udpProm = udpConnectionRequest(
        u.hostname,
        parseInt(u.port || "80"),
        authExp,
        UDP_CONNECTION_REQUEST_PORT,
        debug,
        deviceId,
      ).then(
        () => true,
        () => false,
      );
    } catch {
      // Ignore invalid address
    }
  }
  let status;
  if (connReqJabberId && XMPP_CONFIGURED) {
    status = await xmppConnectionRequest(
      connReqJabberId,
      authExp,
      CONNECTION_REQUEST_TIMEOUT,
      debug,
      deviceId,
    );
  } else {
    status = await httpConnectionRequest(
      connectionRequestUrl,
      authExp,
      CONNECTION_REQUEST_ALLOW_BASIC_AUTH,
      CONNECTION_REQUEST_TIMEOUT,
      debug,
      deviceId,
    );
  }
  if (await udpProm) return "";
  return status;
 }
 export async function awaitSessionStart(
  deviceId: string,
  lastInform: number,
  timeout: number,
 ): Promise<boolean> {
  const now = Date.now();
  const device = await collections.devices.findOne(
    { _id: deviceId },
    { projection: { _lastInform: 1 } },
  );
  const li = (device["_lastInform"] as Date).getTime();
  if (li > lastInform) return true;
  const token = await getToken(`cwmp_session_${deviceId}`);
  if (token?.startsWith("cwmp_session_")) return true;
  if (timeout < 500) return false;
  await new Promise((resolve) => setTimeout(resolve, 500));
  timeout -= Date.now() - now;
  return awaitSessionStart(deviceId, lastInform, timeout);
 }
 export async function awaitSessionEnd(
  deviceId: string,
  timeout: number,
 ): Promise<boolean> {
  const now = Date.now();
  const token = await getToken(`cwmp_session_${deviceId}`);
  if (!token?.startsWith("cwmp_session_")) return true;
  if (timeout < 500) return false;
  await new Promise((resolve) => setTimeout(resolve, 500));
  timeout -= Date.now() - now;
  return awaitSessionEnd(deviceId, timeout);
 }
 function sanitizeTask(task): void {
  task.timestamp = new Date(task.timestamp || Date.now());
  if (task.expiry) {
    if (task.expiry instanceof Date || isNaN(task.expiry))
      task.expiry = new Date(task.expiry);
    else task.expiry = new Date(task.timestamp.getTime() + +task.expiry * 1000);
  }
  const validParamValue = (p): boolean => {
    if (
      !Array.isArray(p) ||
      p.length < 2 ||
      typeof p[0] !== "string" ||
      !p[0].length ||
      !["string", "boolean", "number"].includes(typeof p[1]) ||
      (p[2] != null && typeof p[2] !== "string")
    )
      return false;
    return true;
  };
  switch (task.name) {
    case "getParameterValues":
      if (!Array.isArray(task.parameterNames) || !task.parameterNames.length)
        throw new Error("Missing 'parameterNames' property");
      for (const p of task.parameterNames) {
        if (typeof p !== "string" || !p.length)
          throw new Error(`Invalid parameter name '${p}'`);
      }
      break;
    case "setParameterValues":
      if (!Array.isArray(task.parameterValues) || !task.parameterValues.length)
        throw new Error("Missing 'parameterValues' property");
      for (const p of task.parameterValues) {
        if (!validParamValue(p))
          throw new Error(`Invalid parameter value '${p}'`);
      }
      break;
    case "refreshObject":
      if (typeof task.objectName !== "string")
        throw new Error("Missing 'objectName' property");
      break;
    case "deleteObject":
      if (typeof task.objectName !== "string" || !task.objectName.length)
        throw new Error("Missing 'objectName' property");
      break;
    case "addObject":
      if (task.parameterValues != null) {
        if (!Array.isArray(task.parameterValues))
          throw new Error("Invalid 'parameterValues' property");
        for (const p of task.parameterValues) {
          if (!validParamValue(p))
            throw new Error(`Invalid parameter value '${p}'`);
        }
      }
      break;
    case "download":
      // genieacs-gui sends file ID instead of fileName and fileType
      if (!task.file) {
        if (typeof task.fileType !== "string" || !task.fileType.length)
          throw new Error("Missing 'fileType' property");
        if (typeof task.fileName !== "string" || !task.fileName.length)
          throw new Error("Missing 'fileName' property");
      }
      if (
        task.targetFileName != null &&
        typeof task.targetFileName !== "string"
      )
        throw new Error("Invalid 'targetFileName' property");
      break;
    case "provisions":
      if (
        !Array.isArray(task.provisions) ||
        !task.provisions.every((arr) =>
          arr.every(
            (s) =>
              s == null || ["boolean", "number", "string"].includes(typeof s),
          ),
        )
      )
        throw new Error("Invalid 'provisions' property");
      break;
    case "reboot":
      break;
    case "factoryReset":
      break;
    default:
      throw new Error("Invalid task name");
  }
  return task;
 }
 export async function insertTasks(tasks: any[]): Promise<Task[]> {
  if (tasks && !Array.isArray(tasks)) tasks = [tasks];
  else if (!tasks?.length) return tasks || [];
  for (const task of tasks) {
    sanitizeTask(task);
    if (task.uniqueKey) {
      await collections.tasks.deleteOne({
        device: task.device,
        uniqueKey: task.uniqueKey,
      });
    }
  }
  await collections.tasks.insertMany(tasks);
  for (const task of tasks) task._id = task._id.toString();
  return tasks;
 }
 export async function deleteDevice(deviceId: string): Promise<void> {
  const token = await acquireLock(`cwmp_session_${deviceId}`, 5000);
  if (!token) throw new ResourceLockedError("Device is in session");
  try {
    await Promise.all([
      collections.tasks.deleteMany({ device: deviceId }),
      collections.devices.deleteOne({ _id: deviceId }),
      collections.faults.deleteMany({
        _id: {
          $regex: `^${common.escapeRegExp(deviceId)}\\:`,
        },
      }),
      collections.operations.deleteMany({
        _id: {
          $regex: `^${common.escapeRegExp(deviceId)}\\:`,
        },
      }),
    ]);
  } finally {
    await releaseLock(`cwmp_session_${deviceId}`, token);
  }
 }
 export async function deleteFault(id: string): Promise<void> {
  const deviceId = id.split(":", 1)[0];
  const channel = id.slice(deviceId.length + 1);
  const token = await acquireLock(`cwmp_session_${deviceId}`, 5000);
  if (!token) throw new ResourceLockedError("Device is in session");
  try {
    const proms = [dbDeleteFault(id)];
    if (channel.startsWith("task_"))
      proms.push(deleteTask(new ObjectId(channel.slice(5))));
    await Promise.all(proms);
  } finally {
    await releaseLock(`cwmp_session_${deviceId}`, token);
  }
 }
 export async function deleteResource(
  resource: string,
  id: string,
 ): Promise<void> {
  if (resource === "devices") {
    await deleteDevice(id);
  } else if (resource === "files") {
    await deleteFile(id);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "faults") {
    await deleteFault(id);
  } else if (resource === "provisions") {
    await deleteProvision(id);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "presets") {
    await deletePreset(id);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "virtualParameters") {
    await deleteVirtualParameter(id);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "config") {
    await deleteConfig(id);
    await Promise.all([
      cache.del("ui-local-cache-hash"),
      cache.del("cwmp-local-cache-hash"),
    ]);
  } else if (resource === "permissions") {
    await deletePermission(id);
    await cache.del("ui-local-cache-hash");
  } else if (resource === "users") {
    await deleteUser(id);
    await cache.del("ui-local-cache-hash");
  } else {
    throw new Error(`Unknown resource ${resource}`);
  }
 }
 // TODO Implement validation
 export async function putResource(
  resource: string,
  id: string,
  data: any,
 ): Promise<void> {
  if (resource === "presets") {
    await putPreset(id, data);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "provisions") {
    await putProvision(id, data);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "virtualParameters") {
    await putVirtualParameter(id, data);
    await cache.del("cwmp-local-cache-hash");
  } else if (resource === "config") {
    await putConfig(id, data);
    await Promise.all([
      cache.del("ui-local-cache-hash"),
      cache.del("cwmp-local-cache-hash"),
    ]);
  } else if (resource === "permissions") {
    await putPermission(id, data);
    await cache.del("ui-local-cache-hash");
  } else if (resource === "users") {
    delete data["password"];
    delete data["salt"];
    await putUser(id, data);
    await cache.del("ui-local-cache-hash");
  } else {
    throw new Error(`Unknown resource ${resource}`);
  }
 }
 export function authLocal(
  snapshot: string,
  username: string,
  password: string,
 ): Promise<boolean> {
  return new Promise((resolve, reject) => {
    const users = getUsers(snapshot);
    const user = users[username];
    if (!user?.password) return void resolve(null);
    hashPassword(password, user.salt)
      .then((hash) => {
        if (hash === user.password) resolve(true);
        else resolve(false);
      })
      .catch(reject);
  });
 }
@@ -0,0 +1,80 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 ###
 crypto = require 'crypto'
 exports.parseAuthHeader = (authHeader) ->
  res = {}
  i = authHeader.indexOf(' ')
  res['method'] = authHeader.slice(0, i)
  options = authHeader.slice(i + 1)
  regex = /([a-z]+)="([a-zA-Z0-9\/\.@\-\+,= ]+)"/g
  while (r = regex.exec(options)) != null
    res[r[1]] = r[2]
  regex = /([a-z]+)=([a-zA-Z0-9\/\.@\-\+]+)/g
  while (r = regex.exec(options)) != null
    res[r[1]] = r[2]
  return res
 exports.basic = (username, password) ->
  "Basic #{new Buffer("#{username}:#{password}").toString('base64')}"
 exports.digest = (username, password, uri, httpMethod, body, authHeader) ->
  cnonce = '0a4f113b'
  nc = '00000001'
  if authHeader.qop?
    if authHeader.qop.indexOf(',') != -1
      qop = 'auth' # either auth or auth-int. prefer auth
    else
      qop = authHeader.qop
  ha1 = crypto.createHash('md5')
  ha1.update(username).update(':').update(authHeader.realm).update(':').update(password)
  # TODO support "MD5-sess" algorithm directive
  ha1 = ha1.digest('hex')
  ha2 = crypto.createHash('md5')
  ha2.update(httpMethod).update(':').update(uri)
  if qop == 'auth-int'
    ha2.update(':').update(body)
  ha2 = ha2.digest('hex')
  response = crypto.createHash('md5')
  response.update(ha1).update(':').update(authHeader.nonce)
  if qop?
    response.update(':').update(nc).update(':').update(cnonce).update(':').update(qop)
  response.update(':').update(ha2)
  response = response.digest('hex')
  authString = "Digest username=\"#{username}\""
  authString += ",realm=\"#{authHeader.realm}\""
  authString += ",nonce=\"#{authHeader.nonce}\""
  authString += ",uri=\"#{uri}\""
  authString += ",algorithm=#{authHeader.algorithm}" if authHeader.algorithm?
  authString += ",qop=#{qop}"
  authString += ",nc=#{nc}"
  authString += ",cnonce=\"#{cnonce}\""
  authString += ",response=\"#{response}\""
  authString += ",opaque=\"#{authHeader.opaque}\""
  return authString
@@ -1,178 +0,0 @@
 import { createHash, randomBytes, pbkdf2 } from "node:crypto";
 function parseHeaderFeilds(str: string): Record<string, string> {
  const res = {};
  const parts = str.split(",");
  let part: string;
  while ((part = parts.shift()) != null) {
    const name = part.split("=", 1)[0];
    if (name.length === part.length) {
      if (!part.trim()) continue;
      throw new Error("Unable to parse auth header");
    }
    let value = part.slice(name.length + 1);
    if (!/^\s*"/.test(value)) {
      value = value.trim();
    } else {
      while (!/[^\\]"\s*$/.test(value)) {
        const p = parts.shift();
        if (p == null) throw new Error("Unable to parse auth header");
        value += "," + p;
      }
      try {
        value = JSON.parse(value);
      } catch (error) {
        throw new Error("Unable to parse auth header", { cause: error });
      }
    }
    res[name.trim()] = value;
  }
  return res;
 }
 export function parseAuthorizationHeader(authHeader: string): {
  method: string;
 } {
  authHeader = authHeader.trim();
  const method = authHeader.split(" ", 1)[0];
  const res = { method: method };
  if (method === "Basic") {
    // Inspired by https://github.com/jshttp/basic-auth
    const USER_PASS_REGEX = /^([^:]*):(.*)$/;
    const creds = USER_PASS_REGEX.exec(
      Buffer.from(authHeader.slice(method.length + 1), "base64").toString(),
    );
    if (!creds) throw new Error("Unable to parse auth header");
    res["username"] = creds[1];
    res["password"] = creds[2];
  } else if (method === "Digest") {
    Object.assign(res, parseHeaderFeilds(authHeader.slice(method.length + 1)));
  }
  return res;
 }
 export function parseWwwAuthenticateHeader(
  authHeader: string,
 ): Record<string, string> {
  authHeader = authHeader.trim();
  const method = authHeader.split(" ", 1)[0];
  const res = { method: method };
  Object.assign(res, parseHeaderFeilds(authHeader.slice(method.length + 1)));
  return res;
 }
 export function basic(username: string, password: string): string {
  return "Basic " + Buffer.from(`${username}:${password}`).toString("base64");
 }
 export function digest(
  username: string | Buffer,
  realm: string | Buffer,
  password: string | Buffer,
  nonce: string | Buffer,
  httpMethod: string | Buffer,
  uri: string | Buffer,
  qop?: string | Buffer,
  body?: string | Buffer,
  cnonce?: string | Buffer,
  nc?: string | Buffer,
 ): string {
  const ha1 = createHash("md5");
  ha1.update(username).update(":").update(realm).update(":").update(password);
  // TODO support "MD5-sess" algorithm directive
  const ha1d = ha1.digest("hex");
  const ha2 = createHash("md5");
  ha2.update(httpMethod).update(":").update(uri);
  if (qop === "auth-int") {
    const bodyHash = createHash("md5")
      .update(body || "")
      .digest("hex");
    ha2.update(":").update(bodyHash);
  }
  const ha2d = ha2.digest("hex");
  const hash = createHash("md5");
  hash.update(ha1d).update(":").update(nonce);
  if (qop) {
    hash
      .update(":")
      .update(nc)
      .update(":")
      .update(cnonce)
      .update(":")
      .update(qop);
  }
  hash.update(":").update(ha2d);
  return hash.digest("hex");
 }
 export function solveDigest(
  username: string | Buffer,
  password: string | Buffer,
  uri: string | Buffer,
  httpMethod: string | Buffer,
  body: string | Buffer,
  authHeader: Record<string, string>,
 ): string {
  const cnonce = randomBytes(8).toString("hex");
  const nc = "00000001";
  let qop;
  if (authHeader.qop) {
    if (authHeader.qop.indexOf(",") !== -1) qop = "auth";
    // Either auth or auth-int, prefer auth
    else qop = authHeader.qop;
  }
  const hash = digest(
    username,
    authHeader.realm,
    password,
    authHeader.nonce,
    httpMethod,
    uri,
    qop,
    body,
    cnonce,
    nc,
  );
  let authString = `Digest username="${username}"`;
  authString += `,realm="${authHeader.realm}"`;
  authString += `,nonce="${authHeader.nonce}"`;
  authString += `,uri="${uri}"`;
  if (authHeader.algorithm) authString += `,algorithm=${authHeader.algorithm}`;
  if (qop) authString += `,qop=${qop},nc=${nc},cnonce="${cnonce}"`;
  authString += `,response="${hash}"`;
  if (authHeader.opaque) authString += `,opaque="${authHeader.opaque}"`;
  return authString;
 }
 export function generateSalt(length: number): Promise<string> {
  return new Promise((resolve, reject) => {
    randomBytes(length, (err, rand) => {
      if (err) return void reject(err);
      resolve(rand.toString("hex"));
    });
  });
 }
 export function hashPassword(pass: string, salt: string): Promise<string> {
  return new Promise((resolve, reject) => {
    pbkdf2(pass, salt, 10000, 128, "sha512", (err, hash) => {
      if (err) return void reject(err);
      resolve(hash.toString("hex"));
    });
  });
 }
@@ -1,35 +0,0 @@
 import { collections } from "./db/db.ts";
 import * as config from "./config.ts";
 const CLOCK_SKEW_TOLERANCE = 30000;
 const MAX_CACHE_TTL = +config.get("MAX_CACHE_TTL");
 export async function get(key: string): Promise<string> {
  const res = await collections.cache.findOne({ _id: key });
  return res?.value;
 }
 export async function del(key: string): Promise<void> {
  await collections.cache.deleteOne({ _id: key });
 }
 export async function set(
  key: string,
  value: string,
  ttl: number = MAX_CACHE_TTL,
 ): Promise<void> {
  const timestamp = new Date();
  const expire = new Date(
    timestamp.getTime() + CLOCK_SKEW_TOLERANCE + ttl * 1000,
  );
  await collections.cache.replaceOne(
    { _id: key },
    { value, expire, timestamp },
    { upsert: true },
  );
 }
 export async function pop(key: string): Promise<string> {
  const res = await collections.cache.findOneAndDelete({ _id: key });
  return res.value?.value;
 }
@@ -0,0 +1,50 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 ###
 config = require './config'
 cluster = require 'cluster'
 util = require 'util'
 # Keep current configuration to pass that to workers as environment variables
 ENVIRONMENT_VARIABLES = {}
 for k, v of config.allConfig
  ENVIRONMENT_VARIABLES["GENIEACS_#{k}"] = v
 cluster.on('listening', (worker, address) ->
  util.log("Worker #{worker.process.pid} listening to #{address.address}:#{address.port}")
 )
 cluster.on('exit', (worker, code, signal) ->
  util.log("Worker #{worker.process.pid} died (#{worker.process.exitCode})")
  cluster.fork(ENVIRONMENT_VARIABLES)
 )
 start = (service) ->
  workerProcesses = config.get("#{service.toUpperCase()}_WORKER_PROCESSES")
  cluster.setupMaster({
      exec : 'lib/server',
      args : [service]
  })
  for [0 ... workerProcesses]
    cluster.fork(ENVIRONMENT_VARIABLES)
 exports.start = start
@@ -1,105 +0,0 @@
 import cluster, { Worker } from "node:cluster";
 import { cpus } from "node:os";
 import * as logger from "./logger.ts";
 let respawnTimestamp = 0;
 let crashes: number[] = [];
 function fork(): Worker {
  const w = cluster.fork();
  w.on("error", (err: NodeJS.ErrnoException) => {
    // Avoid exception when attempting to kill the process just as it's exiting
    if (err.code !== "EPIPE") throw err;
    setTimeout(() => {
      if (!w.isDead()) throw err;
    }, 50);
  });
  return w;
 }
 function restartWorker(worker, code, signal): void {
  const msg = {
    message: "Worker died",
    pid: worker.process.pid,
    exitCode: null,
    signal: null,
  };
  if (code != null) msg.exitCode = code;
  if (signal != null) msg.signal = signal;
  logger.error(msg);
  const now = Date.now();
  crashes.push(now);
  let min1 = 0,
    min2 = 0,
    min3 = 0;
  crashes = crashes.filter((n) => {
    if (n > now - 60000) ++min1;
    else if (n > now - 120000) ++min2;
    else if (n > now - 180000) ++min3;
    else return false;
    return true;
  });
  if (min1 > 5 && min2 > 5 && min3 > 5) {
    process.exitCode = 1;
    cluster.removeListener("exit", restartWorker);
    for (const pid in cluster.workers) cluster.workers[pid].kill();
    logger.error({
      message: "Too many crashes, exiting",
      pid: process.pid,
    });
    return;
  }
  respawnTimestamp = Math.max(now, respawnTimestamp + 2000);
  if (respawnTimestamp === now) {
    fork();
    return;
  }
  setTimeout(() => {
    if (process.exitCode) return;
    fork();
  }, respawnTimestamp - now);
 }
 export function start(
  workerCount: number,
  servicePort: number,
  serviceAddress: string,
 ): void {
  cluster.on("listening", (worker, address) => {
    if (
      (address.addressType === 4 || address.addressType === 6) &&
      address.address === serviceAddress &&
      address.port === servicePort
    ) {
      logger.info({
        message: "Worker listening",
        pid: worker.process.pid,
        address: address.address,
        port: address.port,
      });
    }
  });
  cluster.on("exit", restartWorker);
  if (!workerCount) workerCount = Math.max(2, cpus().length);
  for (let i = 0; i < workerCount; ++i) fork();
 }
 export function stop(): void {
  cluster.removeListener("exit", restartWorker);
  for (const pid in cluster.workers) cluster.workers[pid].kill();
 }
 export const worker = cluster.worker;
@@ -0,0 +1,133 @@
 ###
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 buffer = require 'buffer'
 querystring = require 'querystring'
 UNDEFINED_TYPE = '[object Undefined]'
 NULL_TYPE = '[object Null]'
 BOOLEAN_TYPE = '[object Boolean]'
 NUMBER_TYPE = '[object Number]'
 STRING_TYPE = '[object String]'
 ARRAY_TYPE = '[object Array]'
 OBJECT_TYPE = '[object Object]'
 REGEXP_TYPE = '[object RegExp]'
 DATE_TYPE = '[object Date]'
 typeOf = (obj) ->
  Object.prototype.toString.call(obj)
 exports.endsWith = (str, suffix) ->
  str.indexOf(suffix, str.length - suffix.length) isnt -1
 exports.startsWith = (str, prefix) ->
  str.substring(0, prefix.length) == prefix
 exports.arrayToHash = (arr) ->
  hash = {}
  for i in arr
    hash[i[0]] = i[1]
  return hash
 exports.generateDeviceId = (deviceIdStruct) ->
  # Percent escaping function. Escapes everything except alphanumerics and underscore
  esc = (str) ->
    str.replace(/[^A-Za-z0-9_]/g, (chr) ->
      buf = new buffer.Buffer(chr, 'utf8')
      rep = ''
      rep += "%#{b.toString(16).toUpperCase()}" for b in buf
      return rep
    )
  # Guaranteeing globally unique id as defined in TR-069
  if deviceIdStruct['ProductClass']
    return "#{esc(deviceIdStruct['OUI'])}-#{esc(deviceIdStruct['ProductClass'])}-#{esc(deviceIdStruct['SerialNumber'])}"
  return "#{esc(deviceIdStruct['OUI'])}-#{esc(deviceIdStruct['SerialNumber'])}"
 exports.parseDeviceId = (deviceId) ->
  parts = deviceId.split('-')
  ret = {oui : querystring.unescape(parts[0])}
  if parts.length == 3
    ret.productClass = querystring.unescape(parts[1])
    ret.serialNumber = querystring.unescape(parts[2])
  else
    ret.serialNumber = querystring.unescape(parts[1])
  return ret
 exports.extend = (obj, mixin) ->
  obj[name] = method for name, method of mixin
  obj
 exports.flattenObject = (object) ->
  newObj = {}
  f = (obj, prefix) ->
    for k, v of obj
      if typeof(v) is 'object'
        f(v, "#{prefix}#{k}.")
      else
        newObj["#{prefix}#{k}"] = v
  f(object, '')
  return newObj
 exports.getParamValueFromPath = (obj, path) ->
  pp = path.split('.')
  ref = obj
  try
    for p in pp
      ref = ref[p]
    return ref
  catch err
    return undefined
 exports.matchType = (src, dst) ->
  switch typeOf(src)
    when STRING_TYPE
      String(dst)
    when NUMBER_TYPE
      Number(dst)
    when BOOLEAN_TYPE
      v = String(dst).trim().toLowerCase()
      v == 'true' or v == 'on' or v == 'yes' or v == '1'
    else
      dst
 exports.UNDEFINED_TYPE = UNDEFINED_TYPE
 exports.NULL_TYPE = NULL_TYPE
 exports.NUMBER_TYPE = NUMBER_TYPE
 exports.STRING_TYPE = STRING_TYPE
 exports.ARRAY_TYPE = ARRAY_TYPE
 exports.OBJECT_TYPE = OBJECT_TYPE
 exports.REGEXP_TYPE = REGEXP_TYPE
 exports.typeOf = typeOf
@@ -1,129 +0,0 @@
 import { PermissionSet } from "../types.ts";
 import Expression from "./expression.ts";
 export default class Authorizer {
  declare private permissionSets: PermissionSet[];
  declare private validatorCache: WeakMap<
    any,
    (mutationType, mutation, any) => boolean
  >;
  declare private hasAccessCache: Map<string, boolean>;
  declare private getFilterCache: Map<string, Expression>;
  public constructor(permissionSets: PermissionSet[]) {
    this.permissionSets = permissionSets;
    this.validatorCache = new WeakMap();
    this.hasAccessCache = new Map();
    this.getFilterCache = new Map();
  }
  public hasAccess(resourceType: string, access: number): boolean {
    const cacheKey = `${resourceType}-${access}`;
    if (this.hasAccessCache.has(cacheKey))
      return this.hasAccessCache.get(cacheKey);
    let has = false;
    for (const permissionSet of this.permissionSets) {
      for (const perm of permissionSet) {
        if (perm[resourceType]) {
          if (perm[resourceType].access >= access) {
            has = true;
            break;
          }
        }
      }
    }
    this.hasAccessCache.set(cacheKey, has);
    return has;
  }
  public getFilter(resourceType: string, access: number): Expression {
    const cacheKey = `${resourceType}-${access}`;
    if (this.getFilterCache.has(cacheKey))
      return this.getFilterCache.get(cacheKey);
    let filter: Expression = new Expression.Literal(false);
    for (const permissionSet of this.permissionSets) {
      for (const perm of permissionSet) {
        if (perm[resourceType]) {
          if (perm[resourceType].access >= access)
            filter = Expression.or(filter, perm[resourceType].filter);
        }
      }
    }
    this.getFilterCache.set(cacheKey, filter);
    return filter;
  }
  public getValidator(
    resourceType: string,
    resource: unknown,
  ): (mutationType: string, mutation?: any, args?: any) => boolean {
    if (this.validatorCache.has(resource))
      return this.validatorCache.get(resource);
    let validators: Expression = new Expression.Literal(false);
    for (const permissionSet of this.permissionSets) {
      for (const perm of permissionSet) {
        if (
          perm[resourceType] &&
          perm[resourceType].access >= 3 &&
          perm[resourceType].validate
        )
          validators = Expression.or(validators, perm[resourceType].validate);
      }
    }
    const validator = (
      mutationType: string,
      mutation: any,
      any: any,
    ): boolean => {
      const object = {
        mutationType,
        mutation,
        resourceType,
        object: resource,
        options: any,
      };
      const now = Date.now();
      const res = validators.evaluate((exp) => {
        if (exp instanceof Expression.Literal) return exp;
        if (exp instanceof Expression.Parameter) {
          if (exp.path.colon) return new Expression.Literal(null);
          const entry = exp.path.segments[0] as string;
          const paramName = exp.path.slice(1);
          let value = null;
          if (["mutation", "options"].includes(entry)) {
            value = object[entry];
            for (const seg of paramName.segments) {
              if (value == null) break;
              if (typeof value !== "object") value = null;
              else value = value[seg as string];
            }
          } else if (object[entry]) {
            if (paramName.length) value = object[entry][paramName.toString()];
            else value = object[entry];
          }
          return new Expression.Literal(value);
        } else if (exp instanceof Expression.FunctionCall) {
          if (exp.name === "NOW") return new Expression.Literal(now);
        }
        return new Expression.Literal(null);
      }).value;
      return !!res;
    };
    this.validatorCache.set(resource, validator);
    return validator;
  }
  public getPermissionSets(): PermissionSet[] {
    return this.permissionSets;
  }
 }
@@ -1,16 +0,0 @@
 export default function debounce<T>(
  func: (args: T[]) => void,
  timeout: number,
 ): (arg: T) => void {
  let timer: ReturnType<typeof setTimeout>;
  let args: T[] = [];
  return (arg: T) => {
    args.push(arg);
    clearTimeout(timer);
    timer = setTimeout(() => {
      const argscopy = args;
      args = [];
      func(argscopy);
    }, timeout);
  };
 }
@@ -1,6 +0,0 @@
 export class ResourceLockedError extends Error {
  constructor(message: string) {
    super(message);
    this.name = "ResourceLockedError";
  }
 }
@@ -1,291 +0,0 @@
 import {
  Cursor,
  parseExpression,
  stringifyExpression,
 } from "./expression/parser.ts";
 import Path from "./path.ts";
 import { reduce } from "./expression/evaluate.ts";
 export type Value = string | number | boolean | null;
 export abstract class Expression {
  private _string: string;
  abstract map(fn: (e: Expression, i: number) => Expression): Expression;
  abstract mapAsync(
    fn: (e: Expression, i: number) => Promise<Expression>,
  ): Promise<Expression>;
  toString(): string {
    if (!this._string) this._string = stringifyExpression(this);
    return this._string;
  }
  evaluate<T extends Expression>(fn: (e: Expression) => T = (e) => e as T): T {
    return fn(reduce(this.map((e) => e.evaluate(fn))));
  }
  async evaluateAsync<T extends Expression>(
    fn: (e: Expression) => Promise<T>,
  ): Promise<T> {
    return await fn(
      reduce(await this.mapAsync(async (e) => await e.evaluateAsync(fn))),
    );
  }
  static parse(input: string): Expression {
    const cursor = new Cursor(input);
    const exp = parseExpression(cursor);
    if (cursor.charCode) throw new Error("Unexpected character");
    return exp;
  }
  static and(left: Expression, right: Expression): Expression {
    // Flatten same-operator tree into operands
    const operands: Expression[] = [];
    const stack: Expression[] = [right, left];
    while (stack.length) {
      const e = stack.pop()!;
      if (e instanceof Expression.Binary && e.operator === "AND")
        stack.push(e.right, e.left);
      else operands.push(e);
    }
    // Fold literals using three-valued AND logic
    let folded: boolean | null = true;
    let i = 0;
    while (i < operands.length) {
      const e = operands[i];
      if (e instanceof Expression.Literal) {
        operands.splice(i, 1);
        if (e.value == null) folded = folded === false ? false : null;
        else if (!e.value) return new Expression.Literal(false);
        // truthy is identity for AND; discard
      } else {
        i++;
      }
    }
    // Rebuild: folded value + remaining non-literal operands
    if (!operands.length) return new Expression.Literal(folded);
    let result = operands.reduce((a, b) => new Expression.Binary("AND", a, b));
    if (folded === null)
      result = new Expression.Binary(
        "AND",
        new Expression.Literal(null),
        result,
      );
    return result;
  }
  static or(left: Expression, right: Expression): Expression {
    // Flatten same-operator tree into operands
    const operands: Expression[] = [];
    const stack: Expression[] = [right, left];
    while (stack.length) {
      const e = stack.pop()!;
      if (e instanceof Expression.Binary && e.operator === "OR")
        stack.push(e.right, e.left);
      else operands.push(e);
    }
    // Fold literals using three-valued OR logic
    let folded: boolean | null = false;
    let i = 0;
    while (i < operands.length) {
      const e = operands[i];
      if (e instanceof Expression.Literal) {
        operands.splice(i, 1);
        if (e.value == null) folded = folded === true ? true : null;
        else if (e.value) return new Expression.Literal(true);
        // falsy is identity for OR; discard
      } else {
        i++;
      }
    }
    // Rebuild: folded value + remaining non-literal operands
    if (!operands.length) return new Expression.Literal(folded);
    let result = operands.reduce((a, b) => new Expression.Binary("OR", a, b));
    if (folded === null)
      result = new Expression.Binary(
        "OR",
        new Expression.Literal(null),
        result,
      );
    return result;
  }
 }
 // eslint-disable-next-line @typescript-eslint/no-namespace
 export namespace Expression {
  export class Literal extends Expression {
    constructor(public readonly value: Value) {
      super();
    }
    map(): Literal {
      return this;
    }
    async mapAsync(): Promise<Literal> {
      return this;
    }
  }
  export class Parameter extends Expression {
    constructor(public readonly path: Path) {
      super();
    }
    map(): Parameter {
      return this;
    }
    async mapAsync(): Promise<Parameter> {
      return this;
    }
    toString(): string {
      return this.path.toString();
    }
  }
  export class Unary extends Expression {
    constructor(
      public readonly operator: string,
      public readonly operand: Expression,
    ) {
      super();
    }
    map(fn: (e: Expression, i: number) => Expression): Unary {
      const operand = fn(this.operand, 0);
      if (operand === this.operand) return this;
      return Reflect.construct(this.constructor, [this.operator, operand]);
    }
    async mapAsync(
      fn: (e: Expression, i: number) => Promise<Expression>,
    ): Promise<Unary> {
      const operand = await fn(this.operand, 0);
      if (operand === this.operand) return this;
      return Reflect.construct(this.constructor, [this.operator, operand]);
    }
  }
  export class Binary extends Expression {
    constructor(
      public readonly operator: string,
      public readonly left: Expression,
      public readonly right: Expression,
    ) {
      super();
    }
    map(fn: (e: Expression, i: number) => Expression): Binary {
      const left = fn(this.left, 0);
      const right = fn(this.right, 1);
      if (left === this.left && right === this.right) return this;
      return Reflect.construct(this.constructor, [this.operator, left, right]);
    }
    async mapAsync(
      fn: (e: Expression, i: number) => Promise<Expression>,
    ): Promise<Binary> {
      const left = await fn(this.left, 0);
      const right = await fn(this.right, 1);
      if (left === this.left && right === this.right) return this;
      return Reflect.construct(this.constructor, [this.operator, left, right]);
    }
  }
  export class FunctionCall extends Expression {
    constructor(
      public readonly name: string,
      public readonly args: Expression[],
    ) {
      super();
    }
    map(fn: (e: Expression, i: number) => Expression): FunctionCall {
      const args = this.args.map(fn);
      if (args.every((arg, i) => arg === this.args[i])) return this;
      return new FunctionCall(this.name, args);
    }
    async mapAsync(
      fn: (e: Expression, i: number) => Promise<Expression>,
    ): Promise<FunctionCall> {
      const args = await Promise.all(this.args.map(fn));
      if (args.every((arg, i) => arg === this.args[i])) return this;
      return new FunctionCall(this.name, args);
    }
  }
  export class Conditional extends Expression {
    constructor(
      public readonly condition: Expression,
      public readonly then: Expression,
      public readonly otherwise: Expression,
    ) {
      super();
    }
    map(fn: (e: Expression, i: number) => Expression): Conditional {
      const condition = fn(this.condition, 0);
      const then = fn(this.then, 1);
      const otherwise = fn(this.otherwise, 2);
      if (
        condition === this.condition &&
        then === this.then &&
        otherwise === this.otherwise
      )
        return this;
      return new Conditional(condition, then, otherwise);
    }
    async mapAsync(
      fn: (e: Expression, i: number) => Promise<Expression>,
    ): Promise<Conditional> {
      const condition = await fn(this.condition, 0);
      const then = await fn(this.then, 1);
      const otherwise = await fn(this.otherwise, 2);
      if (
        condition === this.condition &&
        then === this.then &&
        otherwise === this.otherwise
      )
        return this;
      return new Conditional(condition, then, otherwise);
    }
  }
 }
 export function extractPaths(exp: Expression): Path[] {
  if (exp instanceof Expression.Parameter) return [exp.path];
  const paths: Path[] = [];
  exp.map((e) => {
    if (e instanceof Expression.Parameter) paths.push(e.path);
    else paths.push(...extractPaths(e));
    return e;
  });
  return paths;
 }
 export function parseList(input: string): Expression[] {
  const CHAR_COMMA = 44;
  const res: Expression[] = [];
  const cursor = new Cursor(input);
  cursor.skipwhitespace();
  if (!cursor.charCode) return res;
  res.push(parseExpression(cursor));
  while (cursor.charCode === CHAR_COMMA) {
    cursor.step();
    res.push(parseExpression(cursor));
  }
  if (cursor.charCode) throw new Error("Unexpected character");
  return res;
 }
 export default Expression;
@@ -1,218 +0,0 @@
 import Expression from "../expression.ts";
 import { likePatternToRegExp } from "./parser.ts";
 function compare(
  a: boolean | number | string,
  b: boolean | number | string,
 ): number {
  if (typeof a === "boolean") a = +a;
  if (typeof b === "boolean") b = +b;
  if (typeof a !== typeof b) return typeof a === "string" ? 1 : -1;
  return a > b ? 1 : a < b ? -1 : 0;
 }
 function toNumber(a: boolean | number | string): number {
  switch (typeof a) {
    case "number":
      return a;
    case "boolean":
      return +a;
    case "string":
      return parseFloat(a) || 0;
  }
 }
 function toString(a: boolean | number | string): string {
  switch (typeof a) {
    case "string":
      return a;
    case "number":
      return a.toString();
    case "boolean":
      return (+a).toString();
  }
 }
 const regExpCache: WeakMap<Expression.Literal, RegExp> = new WeakMap();
 export function reduce(exp: Expression): Expression {
  if (exp instanceof Expression.Literal) return exp;
  if (exp instanceof Expression.Unary) {
    if (exp.operator === "NOT") {
      if (exp.operand instanceof Expression.Literal) {
        if (exp.operand.value == null) return exp.operand;
        return new Expression.Literal(!exp.operand.value);
      } else if (exp.operand instanceof Expression.Unary) {
        if (exp.operand.operator === "NOT") return exp.operand.operand;
      }
    } else if (exp.operator === "IS NULL") {
      if (exp.operand instanceof Expression.Literal) {
        return new Expression.Literal(exp.operand.value == null);
      }
    } else if (exp.operator === "IS NOT NULL") {
      if (exp.operand instanceof Expression.Literal) {
        return new Expression.Literal(exp.operand.value != null);
      }
    }
  } else if (exp instanceof Expression.Binary) {
    if (exp.operator === "AND") {
      return Expression.and(exp.left, exp.right);
    } else if (exp.operator === "OR") {
      return Expression.or(exp.left, exp.right);
    } else if (["=", ">", "<", "<>", ">=", "<="].includes(exp.operator)) {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const c = compare(exp.left.value, exp.right.value);
        switch (exp.operator) {
          case "=":
            return new Expression.Literal(c === 0);
          case ">":
            return new Expression.Literal(c > 0);
          case "<":
            return new Expression.Literal(c < 0);
          case "<>":
            return new Expression.Literal(c !== 0);
          case ">=":
            return new Expression.Literal(c >= 0);
          case "<=":
            return new Expression.Literal(c <= 0);
        }
      }
    } else if (["+", "-", "*", "/"].includes(exp.operator)) {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const a = toNumber(exp.left.value);
        const b = toNumber(exp.right.value);
        switch (exp.operator) {
          case "+":
            return new Expression.Literal(a + b);
          case "-":
            return new Expression.Literal(a - b);
          case "*":
            return new Expression.Literal(a * b);
          case "/":
            return new Expression.Literal(a / b);
        }
      }
    } else if (exp.operator === "%") {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const a = toNumber(exp.left.value);
        const b = Math.trunc(toNumber(exp.right.value));
        if (b === 0) return new Expression.Literal(null);
        return new Expression.Literal(a % b);
      }
    } else if (exp.operator === "||") {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const a = toString(exp.left.value);
        const b = toString(exp.right.value);
        return new Expression.Literal(a + b);
      }
    } else if (exp.operator === "LIKE") {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const s = toString(exp.left.value);
        let r = regExpCache.get(exp.right);
        if (!r) {
          r = likePatternToRegExp(toString(exp.right.value));
          regExpCache.set(exp.right, r);
        }
        return new Expression.Literal(r.test(s));
      }
    } else if (exp.operator === "NOT LIKE") {
      if (exp.left instanceof Expression.Literal && exp.left.value == null)
        return exp.left;
      if (exp.right instanceof Expression.Literal && exp.right.value == null)
        return exp.right;
      if (
        exp.left instanceof Expression.Literal &&
        exp.right instanceof Expression.Literal
      ) {
        const s = toString(exp.left.value);
        let r = regExpCache.get(exp.right);
        if (!r) {
          r = likePatternToRegExp(toString(exp.right.value));
          regExpCache.set(exp.right, r);
        }
        return new Expression.Literal(!r.test(s));
      }
    }
  } else if (exp instanceof Expression.FunctionCall) {
    if (exp.name === "COALESCE") {
      const args = exp.args.filter(
        (arg) => !(arg instanceof Expression.Literal && arg.value == null),
      );
      if (!args.length) return new Expression.Literal(null);
      if (args.length === 1) return args[0];
      if (args[0] instanceof Expression.Literal) return args[0];
      if (args.length !== exp.args.length)
        return new Expression.FunctionCall("COALESCE", args);
    } else if (exp.name === "UPPER") {
      if (exp.args[0] instanceof Expression.Literal) {
        if (exp.args[0].value == null) return exp.args[0];
        const a = toString(exp.args[0].value);
        return new Expression.Literal(a.toUpperCase());
      }
    } else if (exp.name === "LOWER") {
      if (exp.args[0] instanceof Expression.Literal) {
        if (exp.args[0].value == null) return exp.args[0];
        const a = toString(exp.args[0].value);
        return new Expression.Literal(a.toLowerCase());
      }
    } else if (exp.name === "ROUND") {
      let p = 0;
      if (exp.args.length > 1) {
        if (exp.args[1] instanceof Expression.Literal) {
          if (exp.args[1].value == null) return exp.args[1];
          p = Math.trunc(toNumber(exp.args[1].value));
        }
      }
      if (exp.args[0] instanceof Expression.Literal) {
        if (exp.args[0].value == null) return exp.args[0];
        const n = toNumber(exp.args[0].value);
        const d = 10 ** p;
        const m = n * d * (1 + Number.EPSILON);
        return new Expression.Literal(Math.round(m) / d);
      }
    }
  } else if (exp instanceof Expression.Conditional) {
    if (exp.condition instanceof Expression.Literal) {
      if (exp.condition.value) return exp.then;
      return exp.otherwise;
    }
  }
  return exp;
 }
@@ -1,490 +0,0 @@
 import Expression from "../expression.ts";
 import { reduce } from "./evaluate.ts";
 class Indeterminates {
  declare public map: Map<Expression, number>;
  declare public sortedKeys: Expression[];
  public constructor(exp?: Expression) {
    this.map = new Map();
    if (exp) {
      this.map.set(exp, 1);
      this.sortedKeys = [exp];
    } else {
      this.sortedKeys = [];
    }
  }
  public reciprocal(): Indeterminates {
    const res = new Indeterminates();
    res.sortedKeys = this.sortedKeys;
    res.map = new Map();
    for (const [k, v] of this.map) res.map.set(k, 0 - v);
    return res;
  }
  public static multiply(
    indeterminates1: Indeterminates,
    indeterminates2: Indeterminates,
  ): Indeterminates {
    const res = new Indeterminates();
    res.sortedKeys = indeterminates1.sortedKeys.slice();
    res.map = new Map(indeterminates1.map);
    const strMap: Map<string, Expression> = new Map();
    for (const k of res.map.keys()) strMap.set(k.toString(), k);
    for (const [key, val] of indeterminates2.map) {
      const k = strMap.get(key.toString());
      if (!k) {
        res.map.set(key, val);
        res.sortedKeys.push(key);
      } else {
        const v2 = val + res.map.get(k);
        if (!v2) {
          res.map.delete(k);
          res.sortedKeys = res.sortedKeys.filter((s) => s !== k);
        } else {
          res.map.set(k, v2);
        }
      }
    }
    res.sortedKeys.sort((a, b) => {
      const str1 = a.toString();
      const str2 = b.toString();
      if (str1.length !== str2.length) return str2.length - str1.length;
      else if (str1 > str2) return 1;
      else if (str1 < str2) return -1;
      return 0;
    });
    return res;
  }
  public static compare(a: Indeterminates, b: Indeterminates): number {
    if (a.sortedKeys.length !== b.sortedKeys.length)
      return b.sortedKeys.length - a.sortedKeys.length;
    for (let i = 0; i < a.sortedKeys.length; ++i) {
      const k1 = a.sortedKeys[i];
      const w1 = a.map.get(k1);
      const k2 = b.sortedKeys[i];
      const w2 = b.map.get(k2);
      if (w1 !== w2) return w2 - w1;
      if (k1.toString().length > k2.toString().length) return -1;
      else if (k1.toString().length < k2.toString().length) return 1;
      else if (k1.toString() > k2.toString()) return 1;
      else if (k1.toString() < k2.toString()) return -1;
    }
    return 0;
  }
 }
 interface Term {
  indeterminates: Indeterminates;
  coefficientNumerator: bigint;
  coefficientDenominator: bigint;
 }
 function findGcd(a: bigint, b: bigint): bigint {
  while (b !== 0n) {
    const t = b;
    b = a % b;
    a = t;
  }
  return a;
 }
 class Polynomial extends Expression {
  declare public terms: Term[];
  public constructor(terms: Term[]) {
    super();
    this.terms = terms;
  }
  map(): Polynomial {
    return this;
  }
  async mapAsync(): Promise<Polynomial> {
    return this;
  }
  public static simplifyTerms(terms: Term[]): Term[] {
    const ts = terms
      .slice()
      .sort((a: Term, b: Term) =>
        Indeterminates.compare(a.indeterminates, b.indeterminates),
      );
    for (let i = 1; i < ts.length; ++i) {
      const t1 = ts[i - 1];
      const t2 = ts[i];
      if (Indeterminates.compare(t1.indeterminates, t2.indeterminates) === 0) {
        const numerator =
          t1.coefficientNumerator * t2.coefficientDenominator +
          t2.coefficientNumerator * t1.coefficientDenominator;
        const denominator =
          t1.coefficientDenominator * t2.coefficientDenominator;
        const gcd = findGcd(numerator, denominator);
        ts[i] = {
          indeterminates: t2.indeterminates,
          coefficientNumerator: numerator / gcd,
          coefficientDenominator: denominator / gcd,
        };
        ts[i - 1] = {
          indeterminates: t1.indeterminates,
          coefficientNumerator: 0n,
          coefficientDenominator: t1.coefficientDenominator,
        };
      }
    }
    return ts.filter((v) => v.coefficientNumerator !== 0n);
  }
  public static fromIndeterminate(indeterminate: Expression): Polynomial {
    const indeterminates = new Indeterminates(indeterminate);
    const terms = [
      {
        indeterminates: indeterminates,
        coefficientNumerator: 1n,
        coefficientDenominator: 1n,
      },
    ];
    return new Polynomial(terms);
  }
  public static fromConstant(constant: number): Polynomial {
    const [int, frac] = Math.abs(constant).toString(2).split(".", 2);
    let numerator = BigInt("0b" + int);
    if (constant < 0) numerator = numerator / -1n;
    let denominator = 1n;
    if (frac) {
      denominator = 2n ** BigInt(frac.length);
      numerator = numerator * denominator + BigInt("0b" + frac);
    }
    const terms = [
      {
        indeterminates: new Indeterminates(),
        coefficientNumerator: numerator,
        coefficientDenominator: denominator,
      },
    ];
    return new Polynomial(terms);
  }
  public negation(): Polynomial {
    const terms = this.terms.map((t) => ({
      indeterminates: t.indeterminates,
      coefficientNumerator: t.coefficientNumerator * -1n,
      coefficientDenominator: t.coefficientDenominator,
    }));
    return new Polynomial(terms);
  }
  public reciprocal(): Polynomial {
    const terms = this.terms.map((t) => ({
      indeterminates: t.indeterminates.reciprocal(),
      coefficientNumerator: t.coefficientDenominator,
      coefficientDenominator: t.coefficientNumerator,
    }));
    return new Polynomial(terms);
  }
  public constant(): Polynomial {
    const terms = this.terms.filter((t) => !t.indeterminates.sortedKeys.length);
    return new Polynomial(terms);
  }
  public add(rhs: Polynomial): Polynomial {
    return new Polynomial(
      Polynomial.simplifyTerms(this.terms.concat(rhs.terms)),
    );
  }
  public subtract(rhs: Polynomial): Polynomial {
    return this.add(rhs.negation());
  }
  public multiply(rhs: Polynomial): Polynomial {
    const terms: Term[] = [];
    for (const t1 of this.terms) {
      for (const t2 of rhs.terms) {
        const numerator = t1.coefficientNumerator * t2.coefficientNumerator;
        const denominator =
          t1.coefficientDenominator * t2.coefficientDenominator;
        const gcd = findGcd(numerator, denominator);
        terms.push({
          indeterminates: Indeterminates.multiply(
            t1.indeterminates,
            t2.indeterminates,
          ),
          coefficientNumerator: numerator / gcd,
          coefficientDenominator: denominator / gcd,
        });
      }
    }
    return new Polynomial(Polynomial.simplifyTerms(terms));
  }
  public divide(rhs: Polynomial): Polynomial {
    return this.multiply(rhs.reciprocal());
  }
  public toExpression(): Expression {
    const add: Expression[] = [];
    for (const t of this.terms) {
      const coefficient =
        Number(t.coefficientNumerator) / Number(t.coefficientDenominator);
      const mul: Expression[] = [];
      if (t.indeterminates.sortedKeys.length) {
        for (const k of t.indeterminates.sortedKeys) {
          const w = t.indeterminates.map.get(k);
          for (let i = Math.abs(w); i > 0; --i) {
            if (w > 0) mul.push(k);
            else
              mul.push(
                new Expression.Binary("/", new Expression.Literal(1), k),
              );
          }
        }
        if (coefficient !== 1) mul.push(new Expression.Literal(coefficient));
        while (mul.length > 1) {
          const r = mul.pop();
          const l = mul.pop();
          mul.push(new Expression.Binary("*", l, r));
        }
        add.push(mul[0]);
      } else {
        add.push(new Expression.Literal(coefficient));
      }
    }
    while (add.length > 1) {
      const r = add.pop();
      const l = add.pop();
      add.push(new Expression.Binary("+", l, r));
    }
    if (!add.length) return new Expression.Literal(0);
    return add[0];
  }
 }
 const SWAPPED_OPS = {
  "=": "=",
  "<>": "<>",
  ">": "<",
  ">=": "<=",
  "<": ">",
  "<=": ">=",
 };
 function cartesianProduct<T>(arrays: T[][]): T[][] {
  return arrays.reduce<T[][]>(
    (acc, cur) => acc.flatMap((a) => cur.map((item) => [...a, item])),
    [[]],
  );
 }
 function toPolynomial(e: Expression): Polynomial {
  if (e instanceof Polynomial) return e;
  if (e instanceof Expression.Literal) {
    if (e.value == null) return null;
    if (typeof e.value === "number") return Polynomial.fromConstant(e.value);
    if (typeof e.value === "string")
      return Polynomial.fromConstant(parseFloat(e.value) || 0);
    if (typeof e.value === "boolean") return Polynomial.fromConstant(+e.value);
  }
  return Polynomial.fromIndeterminate(e);
 }
 function fromPolynomial(e: Expression): Expression {
  if (e instanceof Polynomial) return e.toExpression();
  if (e instanceof Expression.Conditional) return e.map(fromPolynomial);
  return e;
 }
 function normalizeCallback(exp: Expression): Expression {
  if (exp instanceof Expression.FunctionCall) {
    if (exp.name === "COALESCE") {
      let e: Expression = new Expression.Literal(null);
      for (let i = exp.args.length - 1; i >= 0; --i) {
        e = new Expression.Conditional(
          normalizeCallback(new Expression.Unary("IS NOT NULL", exp.args[i])),
          exp.args[i],
          e,
        );
      }
      return normalizeCallback(e);
    }
  }
  if (exp instanceof Expression.Conditional) {
    let e = exp;
    if (e.condition instanceof Polynomial)
      e = new Expression.Conditional(
        e.condition.toExpression(),
        e.then,
        e.otherwise,
      );
    if (e.then instanceof Expression.Conditional) {
      e = new Expression.Conditional(
        Expression.and(e.condition, e.then.condition),
        e.then.then,
        new Expression.Conditional(e.condition, e.then.otherwise, e.otherwise),
      );
    }
    return e;
  }
  const combs: [Expression, Expression][][] = [];
  const callback: (e: Expression) => Expression = (e) => {
    if (e instanceof Expression.Conditional) {
      combs[combs.length - 1].push([e.condition, e.then]);
      callback(e.otherwise);
    } else {
      combs[combs.length - 1].push([new Expression.Literal(true), e]);
    }
    return e;
  };
  exp.map((e) => {
    combs.push([]);
    callback(e);
    return e;
  });
  if (combs.some((a) => a.length > 1)) {
    let res: Expression = new Expression.Literal(null);
    for (const p of cartesianProduct(combs).reverse()) {
      let condition: Expression = new Expression.Literal(true);
      const e = reduce(
        normalizeCallback(
          exp.map((_, i) => {
            condition = Expression.and(condition, p[i][0]);
            return p[i][1];
          }),
        ),
      );
      if (!(condition instanceof Expression.Literal))
        res = new Expression.Conditional(condition, e, res);
      else if (condition.value) res = e;
    }
    return res;
  }
  if (exp instanceof Expression.Binary) {
    if (["+", "-", "*", "/"].includes(exp.operator)) {
      const lhs = toPolynomial(exp.left);
      const rhs = toPolynomial(exp.right);
      if (lhs == null || rhs == null) return new Expression.Literal(null);
      if (exp.operator === "+") return lhs.add(rhs);
      if (exp.operator === "-") return lhs.subtract(rhs);
      if (exp.operator === "*") return lhs.multiply(rhs);
      if (exp.operator === "/") return lhs.divide(rhs);
    } else if ([">", ">=", "<", "<=", "=", "<>"].includes(exp.operator)) {
      let lhs: Polynomial, rhs: Polynomial;
      if (exp.left instanceof Polynomial) lhs = exp.left;
      else if (exp.left instanceof Expression.Literal) {
        if (exp.left.value == null) return exp.left;
        else if (typeof exp.left.value === "number")
          lhs = Polynomial.fromConstant(exp.left.value);
      }
      if (exp.right instanceof Polynomial) rhs = exp.right;
      else if (exp.right instanceof Expression.Literal) {
        if (exp.right.value == null) return exp.right;
        else if (typeof exp.right.value === "number")
          rhs = Polynomial.fromConstant(exp.right.value);
      }
      if (lhs || rhs) {
        if (!lhs) lhs = Polynomial.fromIndeterminate(exp.left);
        if (!rhs) rhs = Polynomial.fromIndeterminate(exp.right);
        lhs = lhs.subtract(rhs);
        rhs = lhs.constant().negation();
        lhs = lhs.add(rhs);
        if (!lhs.terms.length) {
          const l = lhs.toExpression() as Expression.Literal;
          const r = rhs.toExpression() as Expression.Literal;
          if (exp.operator === "=")
            return new Expression.Literal(l.value === r.value);
          else if (exp.operator === "<>")
            return new Expression.Literal(l.value !== r.value);
          else if (exp.operator === ">")
            return new Expression.Literal(l.value > r.value);
          else if (exp.operator === ">=")
            return new Expression.Literal(l.value >= r.value);
          else if (exp.operator === "<")
            return new Expression.Literal(l.value < r.value);
          else if (exp.operator === "<=")
            return new Expression.Literal(l.value <= r.value);
          else throw new Error("Invalid operator");
        }
        let flipOp = 1;
        const n = lhs.terms[0].coefficientNumerator;
        const d = lhs.terms[0].coefficientDenominator;
        if (n < 0n || d < 0n) flipOp *= -1;
        const reciprocal = new Polynomial([
          {
            indeterminates: new Indeterminates(),
            coefficientNumerator: d,
            coefficientDenominator: n,
          },
        ]);
        lhs = lhs.multiply(reciprocal);
        rhs = rhs.multiply(reciprocal);
        const keys = lhs.terms[0].indeterminates.sortedKeys;
        let invert = lhs.terms[0].indeterminates.map.get(keys[0]) < 0 ? -1 : 0;
        for (const t of lhs.terms)
          for (const v of t.indeterminates.map.values()) invert += v;
        if (invert < 0) {
          flipOp *= -1;
          lhs = lhs.reciprocal();
          rhs = rhs.reciprocal();
        }
        if (flipOp > 0) exp = new Expression.Binary(exp.operator, lhs, rhs);
        else exp = new Expression.Binary(SWAPPED_OPS[exp.operator], lhs, rhs);
      }
    }
  }
  // Restore polynomial expressions
  exp = exp.map(fromPolynomial);
  return exp;
 }
 export default function normalize(exp: Expression): Expression {
  return fromPolynomial(exp.evaluate(normalizeCallback));
 }
@@ -1,160 +0,0 @@
 import { complement } from "espresso-iisojs";
 import Expression from "../expression.ts";
 import normalize from "./normalize.ts";
 import { Clause, SynthContext } from "./synth.ts";
 import Path from "../path.ts";
 type Bookmark = Record<string, null | boolean | number | string>;
 type Minterm = number[];
 export function toBookmark(
  sort: Record<string, number>,
  row: unknown,
 ): Bookmark {
  const bookmark: Bookmark = {};
  for (const param of Object.keys(sort)) {
    let v = row[param];
    if (v != null && typeof v === "object") v = v.value?.[0];
    bookmark[param] = v;
  }
  return bookmark;
 }
 export function bookmarkToExpression(
  bookmark: Bookmark,
  sort: Record<string, number>,
 ): Expression {
  return Object.entries(sort)
    .reverse()
    .reduce(
      (cur, kv) => {
        const [param, asc] = kv;
        const p = new Expression.Parameter(Path.parse(param));
        const b = new Expression.Literal(bookmark[param]);
        if (asc < 0) {
          if (bookmark[param] == null) {
            return Expression.or(
              new Expression.Unary("IS NOT NULL", p),
              Expression.and(new Expression.Unary("IS NULL", p), cur),
            );
          }
          return new Expression.Binary(
            "OR",
            new Expression.Binary(">", p, b),
            new Expression.Binary("AND", new Expression.Binary("=", p, b), cur),
          );
        } else {
          let f: Expression = new Expression.Unary("IS NULL", p);
          if (bookmark[param] == null) return Expression.and(f, cur);
          f = Expression.or(f, new Expression.Binary("<", p, b));
          return Expression.or(
            f,
            Expression.and(new Expression.Binary("=", p, b), cur),
          );
        }
      },
      new Expression.Literal(true) as Expression,
    );
 }
 function getCover(
  context: SynthContext,
  minterm: Minterm,
  allSort: [string, number][],
 ): Minterm[] {
  if (!allSort.length) return [[]];
  const [param, sort] = allSort[0];
  const cov: Minterm = [];
  const nextCov: Minterm = [];
  if (sort > 0) {
    const lhs = new Clause.Exp(new Expression.Parameter(Path.parse(param)));
    const isNull = context.getVar(new Clause.IsNull(lhs));
    cov.push((isNull << 2) ^ 2);
  }
  for (const m of minterm) {
    const clause = context.getClause(m >>> 2);
    if (clause instanceof Clause.IsNull) {
      if (!(clause.operand instanceof Clause.Exp)) continue;
      if (!(clause.operand.exp instanceof Expression.Parameter)) continue;
      if (clause.operand.exp.path.toString() !== param) continue;
      nextCov.push(m);
      if (sort < 0 && m & 1) cov.push(m, m ^ 1);
    } else if (clause instanceof Clause.Compare) {
      if (!(clause.lhs instanceof Clause.Exp)) continue;
      if (!(clause.lhs.exp instanceof Expression.Parameter)) continue;
      if (clause.lhs.exp.path.toString() !== param) continue;
      if (!(m & 1) && sort > 0) continue;
      nextCov.push(m);
      const negate = (m ^ (m >> 1)) & 1;
      if (sort > 0) {
        if (
          (clause.op === "=" && !negate) ||
          (clause.op === ">" && !negate) ||
          (clause.op === "<" && negate)
        ) {
          const c = new Clause.Compare(clause.lhs, ">", clause.rhs);
          const v = context.getVar(c);
          cov.push((v << 2) ^ 3);
        }
      } else if (sort < 0) {
        if (
          (clause.op === "=" && !negate) ||
          (clause.op === ">" && negate) ||
          (clause.op === "<" && !negate)
        ) {
          const c = new Clause.Compare(clause.lhs, "<", clause.rhs);
          const v = context.getVar(c);
          cov.push((v << 2) ^ 0);
        }
      }
    }
  }
  const next = getCover(context, minterm, allSort.slice(1));
  return [cov, ...next.map((n) => [...nextCov, ...n])];
 }
 export function paginate(
  fetched: Expression,
  toFetch: Expression,
  sort: Record<string, number>,
 ): [Expression, Expression] {
  fetched = normalize(fetched);
  if (fetched instanceof Expression.Literal && !fetched.value)
    return [new Expression.Literal(false), toFetch];
  toFetch = normalize(toFetch);
  if (toFetch instanceof Expression.Literal && !toFetch.value)
    return [new Expression.Literal(false), toFetch];
  const synth1 = Clause.fromExpression(fetched);
  const synth2 = Clause.fromExpression(toFetch);
  const context = new SynthContext();
  const expr1Minterms = synth1.getMinterms(context, 0b100);
  const expr2MintermsC = synth2.getMinterms(context, 0b011);
  const gaps = context.sanitizeMinterms(
    complement([
      ...expr1Minterms,
      ...expr2MintermsC,
      ...context.getDcSet([...expr1Minterms, ...expr2MintermsC]),
    ]),
  );
  const cover = gaps.flatMap((m) => getCover(context, m, Object.entries(sort)));
  const minterms1 = context.minimize(complement([...cover, ...expr2MintermsC]));
  const minterms2 = context.minimize(
    complement([...minterms1, ...expr2MintermsC]),
  );
  return [context.toExpression(minterms1), context.toExpression(minterms2)];
 }
@@ -1,557 +0,0 @@
 import Path from "../path.ts";
 import Expression from "../expression.ts";
 export class Cursor {
  input: string;
  pos: number;
  boundryStack: number[][];
  boundry: number[];
  charCode: number;
  constructor(input: string) {
    this.input = input;
    this.pos = 0;
    this.boundryStack = [];
    this.boundry = [];
    this.charCode = input.charCodeAt(0) || 0;
  }
  fork(): Cursor {
    const cursor = new Cursor(this.input);
    cursor.pos = this.pos;
    cursor.boundryStack = this.boundryStack.slice();
    cursor.boundry = this.boundry;
    cursor.charCode = this.charCode;
    return cursor;
  }
  sync(cursor: Cursor): void {
    this.pos = cursor.pos;
    this.boundryStack = cursor.boundryStack.slice();
    this.boundry = cursor.boundry;
    this.charCode = cursor.charCode;
  }
  read(cur: Cursor): string {
    return this.input.slice(this.pos, cur.pos);
  }
  step(): Cursor {
    if (this.charCode === 0) return this;
    ++this.pos;
    this.charCode = this.input.charCodeAt(this.pos) || 0;
    if (this.boundry.includes(this.charCode)) this.charCode = 0;
    return this;
  }
  walk(callback: (charCode: number) => boolean): Cursor {
    while (this.charCode && callback(this.charCode)) this.step();
    return this;
  }
  descend(chars: number[], override = true): void {
    this.boundryStack.push(this.boundry);
    if (override) this.boundry = chars;
    else this.boundry = [...this.boundry, ...chars];
    this.charCode = this.input.charCodeAt(this.pos) || 0;
    if (this.boundry.includes(this.charCode)) this.charCode = 0;
  }
  ascend(): void {
    if (!this.boundryStack.length) throw new Error("Unmatched boundry");
    this.boundry = this.boundryStack.pop();
    this.charCode = this.input.charCodeAt(this.pos) || 0;
  }
  skipwhitespace(): Cursor {
    return this.walk((c) => c <= 32);
  }
 }
 const CHAR_SINGLE_QUOTE = 39;
 const CHAR_DOUBLE_QUOTE = 34;
 const CHAR_OPEN_PAREN = 40;
 const CHAR_CLOSE_PAREN = 41;
 const CHAR_COMMA = 44;
 const CHAR_PERIOD = 46;
 const CHAR_COLON = 58;
 const CHAR_OPEN_BRACKET = 91;
 const CHAR_BACKSLASH = 92;
 const CHAR_CLOSE_BRACKET = 93;
 const BINARY_OPERATORS = [
  ">=",
  "<=",
  "<>",
  "=",
  ">",
  "<",
  "LIKE",
  "NOT LIKE",
  "AND",
  "OR",
  "*",
  "/",
  "%",
  "||",
  "-",
  "+",
 ];
 const PRECEDENCE = {
  OR: 10,
  AND: 11,
  NOT: 12,
  "=": 20,
  "<>": 20,
  ">": 20,
  ">=": 20,
  "<": 20,
  "<=": 20,
  LIKE: 20,
  "NOT LIKE": 20,
  "IS NULL": 20,
  "IS NOT NULL": 20,
  "||": 30,
  "+": 31,
  "-": 31,
  "*": 32,
  "/": 32,
  "%": 32,
 };
 function* range(s: number, e: number): Generator<number> {
  for (let i = s; i < e; i++) yield i;
 }
 const PATH_CHARS = new Set<number>([
  ...range(65, 91),
  ...range(97, 123),
  ...range(48, 58),
  95,
  45,
  42,
  123,
  125,
 ]);
 function findOperator(cursor: Cursor): string {
  cursor.skipwhitespace();
  let found = "";
  let foundCursor = cursor;
  let operators = [...BINARY_OPERATORS, "IS NULL", "IS NOT NULL"];
  for (let i = 0; operators.length && cursor.charCode; ++i) {
    let c = cursor.charCode;
    cursor.step();
    if (c >= 97 && c <= 122) c -= 32;
    if (c <= 32) {
      cursor.skipwhitespace();
      c = 32;
    }
    operators = operators.filter((o) => {
      if (o.charCodeAt(i) !== c) return false;
      if (o.length === i + 1) {
        found = o;
        foundCursor = cursor.fork();
      }
      return true;
    });
  }
  if (found) cursor.sync(foundCursor);
  return found;
 }
 // Turn escaped characters into real ones (e.g. "\\n" becomes "\n").
 function interpretEscapes(str): string {
  const escapes = {
    b: "\b",
    f: "\f",
    n: "\n",
    r: "\r",
    t: "\t",
  };
  return str.replace(/\\(u[0-9a-fA-F]{4}|[^u])/g, (_, escape) => {
    const type = escape.charAt(0);
    const hex = escape.slice(1);
    if (type === "u") return String.fromCharCode(parseInt(hex, 16));
    if (escapes.hasOwnProperty(type)) return escapes[type];
    return type;
  });
 }
 export function parseExpression(cursor: Cursor, presedence = 0): Expression {
  cursor.skipwhitespace();
  const char = cursor.charCode;
  let lhs: Expression;
  if (char === CHAR_OPEN_PAREN) {
    cursor.step();
    cursor.descend([CHAR_CLOSE_PAREN]);
    lhs = parseExpression(cursor, 0);
    cursor.ascend();
    cursor.skipwhitespace();
    if (cursor.charCode !== CHAR_CLOSE_PAREN)
      throw new Error("Expected ')'" + cursor.pos + " " + cursor.charCode);
    cursor.step();
  } else if (char === CHAR_DOUBLE_QUOTE) {
    cursor.step();
    const cursor2 = cursor.fork();
    for (
      cursor2.descend([]);
      cursor2.charCode !== CHAR_DOUBLE_QUOTE;
      cursor2.step()
    ) {
      if (!cursor2.charCode) throw new Error("Unterminated string");
      if (cursor2.charCode === CHAR_BACKSLASH) cursor2.step();
    }
    cursor2.ascend();
    const str = cursor.read(cursor2);
    cursor.sync(cursor2);
    cursor.step();
    return new Expression.Literal(interpretEscapes(str));
  } else if (char === CHAR_SINGLE_QUOTE) {
    cursor.step();
    const cursor2 = cursor.fork();
    for (
      cursor2.descend([]);
      cursor2.charCode !== CHAR_SINGLE_QUOTE;
      cursor2.step()
    ) {
      if (!cursor2.charCode) throw new Error("Unterminated string");
      if (cursor2.charCode === CHAR_BACKSLASH) cursor2.step();
    }
    cursor2.ascend();
    const str = cursor.read(cursor2);
    cursor.sync(cursor2);
    cursor.step();
    return new Expression.Literal(str.replaceAll("''", "'"));
  } else {
    const cursor2 = cursor.fork();
    cursor.walk(
      (c) => c > 32 && c !== CHAR_OPEN_PAREN && c !== CHAR_OPEN_BRACKET,
    );
    const token = cursor2.read(cursor);
    if (!token) throw new Error("Invalid expression");
    if (/^true$/i.test(token)) lhs = new Expression.Literal(true);
    else if (/^false$/i.test(token)) lhs = new Expression.Literal(false);
    else if (/^null$/i.test(token)) lhs = new Expression.Literal(null);
    else if (/^-?(0|[1-9][0-9]*)([.][0-9]+)?([eE][+-]?[0-9]+)?$/.test(token))
      lhs = new Expression.Literal(Number(token));
    else if (/^not$/i.test(token)) {
      lhs = new Expression.Unary(
        "NOT",
        parseExpression(cursor, PRECEDENCE["NOT"]),
      );
    } else if (/^case$/i.test(token)) {
      const pairs: [Expression, Expression][] = [];
      for (;;) {
        cursor.skipwhitespace();
        const whenStr = cursor.fork().read(cursor.walk((c) => c > 32));
        if (/^else$/i.test(whenStr)) {
          lhs = parseExpression(cursor);
          continue;
        }
        if (/^end$/i.test(whenStr)) break;
        else if (lhs) throw new Error("Expected END");
        if (!/^when$/i.test(whenStr)) throw new Error("Expected WHEN");
        const condition = parseExpression(cursor);
        cursor.skipwhitespace();
        const thenStr = cursor.fork().read(cursor.walk((c) => c > 32));
        if (!/^then$/i.test(thenStr)) throw new Error("Expected THEN");
        const then = parseExpression(cursor);
        pairs.push([condition, then]);
      }
      if (!lhs) lhs = new Expression.Literal(null);
      while (pairs.length) {
        const [condition, then] = pairs.pop();
        lhs = new Expression.Conditional(condition, then, lhs);
      }
    } else if (cursor.charCode === CHAR_OPEN_PAREN) {
      cursor.step();
      cursor.descend([CHAR_CLOSE_PAREN]);
      cursor.skipwhitespace();
      const args = [] as Expression[];
      while (cursor.charCode) {
        cursor.descend([CHAR_COMMA], false);
        const e = parseExpression(cursor);
        args.push(e);
        cursor.ascend();
        cursor.skipwhitespace();
        if ((cursor.charCode as number) !== CHAR_COMMA) break;
        cursor.step();
      }
      cursor.ascend();
      cursor.step();
      lhs = new Expression.FunctionCall(token, args);
    } else {
      const p = parsePath(cursor2);
      lhs = new Expression.Parameter(p);
      cursor.sync(cursor2);
    }
  }
  for (;;) {
    cursor.skipwhitespace();
    const cursor2 = cursor.fork();
    const op = findOperator(cursor2);
    const p = PRECEDENCE[op];
    if (p <= presedence) return lhs;
    if (op === "IS NULL") {
      lhs = new Expression.Unary("IS NULL", lhs);
    } else if (op === "IS NOT NULL") {
      lhs = new Expression.Unary("IS NOT NULL", lhs);
    } else if (BINARY_OPERATORS.includes(op)) {
      lhs = new Expression.Binary(op, lhs, parseExpression(cursor2, p));
    } else if (op) throw new Error("Unrecognized operator: " + op);
    else break;
    cursor.sync(cursor2);
  }
  return lhs;
 }
 // Parse old-format alias value: unquoted, double-quoted, or single-quoted.
 // Returns the parsed string value. Cursor is advanced past the value.
 function parseOldAliasValue(cursor: Cursor): string {
  cursor.walk((c) => c <= 32); // skip leading whitespace
  const c = cursor.charCode;
  if (c === CHAR_DOUBLE_QUOTE) {
    // Double-quoted: use JSON.parse semantics (handles \", \n, \uXXXX etc.)
    // Descend with no boundaries so commas/brackets inside quotes are not
    // treated as terminators.
    const start = cursor.pos;
    cursor.step();
    cursor.descend([]);
    while (cursor.charCode) {
      if (cursor.charCode === CHAR_BACKSLASH) {
        cursor.step();
        if (!cursor.charCode) break;
      } else if (cursor.charCode === CHAR_DOUBLE_QUOTE) {
        cursor.ascend();
        cursor.step();
        return JSON.parse(cursor.input.slice(start, cursor.pos)) as string;
      }
      cursor.step();
    }
    throw new Error("Unterminated string");
  }
  if (c === CHAR_SINGLE_QUOTE) {
    // Single-quoted: strip quotes, no escape processing
    cursor.step();
    cursor.descend([]);
    const start = cursor.pos;
    while (cursor.charCode && cursor.charCode !== CHAR_SINGLE_QUOTE)
      cursor.step();
    if (!cursor.charCode) throw new Error("Unterminated string");
    const value = cursor.input.slice(start, cursor.pos);
    cursor.ascend();
    cursor.step();
    return value;
  }
  // Unquoted: read until , or ] (respecting boundary), trim result
  const start = cursor.pos;
  cursor.walk(() => true);
  return cursor.input.slice(start, cursor.pos).trim();
 }
 // Parse old-format alias content: key:value,key:value,...
 // Cursor should be positioned at the start of bracket content
 // (after '[', with ']' set as boundary).
 // Returns an Expression: Binary("AND", ...) chain of Binary("=", param, literal).
 // Empty content returns Literal(true).
 function parseOldAlias(cursor: Cursor): Expression {
  cursor.skipwhitespace();
  if (!cursor.charCode) return new Expression.Literal(true);
  let result: Expression | null = null;
  for (;;) {
    cursor.skipwhitespace();
    // Parse key as a path (supports nested aliases via parsePath).
    // Add colon and comma as boundaries so the path stops there.
    cursor.descend([CHAR_COLON, CHAR_COMMA], false);
    const keyPath = parsePath(cursor);
    cursor.ascend();
    // After ascend, check for the colon separator
    cursor.skipwhitespace();
    if ((cursor.charCode as number) !== CHAR_COLON)
      throw new Error("Expected ':'");
    cursor.step();
    // Parse value (reads until boundary , or ])
    cursor.descend([CHAR_COMMA], false);
    const value = parseOldAliasValue(cursor);
    cursor.ascend();
    const pair = new Expression.Binary(
      "=",
      new Expression.Parameter(keyPath),
      new Expression.Literal(value),
    );
    result = result ? new Expression.Binary("AND", result, pair) : pair;
    cursor.skipwhitespace();
    if ((cursor.charCode as number) !== CHAR_COMMA) break;
    cursor.step();
  }
  return result;
 }
 export function parsePath(cur: Cursor): Path {
  const segments: (string | Expression)[] = [];
  let colon = 0;
  const cur2 = cur.fork();
  for (;;) {
    let char = cur2.charCode;
    let exp: Expression;
    if (char === CHAR_OPEN_BRACKET) {
      if (cur2.pos !== cur.pos) throw new Error("Invalid path");
      cur2.step();
      cur2.descend([CHAR_CLOSE_BRACKET]);
      // Try parsing as expression first
      const cur3 = cur2.fork();
      let err: unknown;
      try {
        exp = parseExpression(cur3);
      } catch (e) {
        err = e;
      }
      // Fall back to old alias format if not a valid expression or produced
      // a bare Parameter (old format like [a:1] mis-parsed as a path).
      if (err || exp instanceof Expression.Parameter) {
        exp = parseOldAlias(cur2);
      } else {
        cur2.sync(cur3);
      }
      cur2.ascend();
      char = cur2.charCode;
      if (char !== CHAR_CLOSE_BRACKET) throw new Error("Expected ']'");
      char = cur2.step().charCode;
    }
    if (!PATH_CHARS.has(char)) {
      if (cur2.pos <= cur.pos) throw new Error("Invalid path");
      if (char === CHAR_COLON) {
        if (colon) throw new Error("Multiple colons");
        colon = segments.length + 1;
      }
      if (exp) segments.push(exp);
      else segments.push(cur.read(cur2));
      if (char !== CHAR_PERIOD && char !== CHAR_COLON) break;
      cur2.step();
      cur.sync(cur2);
      continue;
    }
    if (exp) throw new Error("Invalid path");
    cur2.step();
  }
  cur.sync(cur2);
  if (colon) colon = segments.length - colon;
  return new Path(segments, colon);
 }
 export function stringifyExpression(exp: Expression, level = 0): string {
  function wrap(e: string, op: string): string {
    if (PRECEDENCE[op] <= level) return `(${e})`;
    else return e;
  }
  if (exp instanceof Expression.Literal) {
    if (exp.value == null) return "NULL";
    if (exp.value === true) return "TRUE";
    if (exp.value === false) return "FALSE";
    return JSON.stringify(exp.value);
  } else if (exp instanceof Expression.Unary) {
    if (exp.operator === "NOT") {
      return wrap(
        `NOT ${stringifyExpression(exp.operand, PRECEDENCE[exp.operator])}`,
        "NOT",
      );
    } else if (exp.operator === "IS NULL" || exp.operator === "IS NOT NULL") {
      return wrap(
        `${stringifyExpression(exp.operand, PRECEDENCE[exp.operator])} ${exp.operator}`,
        exp.operator,
      );
    }
  } else if (exp instanceof Expression.Binary) {
    const op = exp.operator;
    if (!(op in PRECEDENCE)) throw new Error("Invalid operator");
    return wrap(
      `${stringifyExpression(exp.left, PRECEDENCE[op] - 1)} ${exp.operator} ${stringifyExpression(exp.right, PRECEDENCE[op])}`,
      op,
    );
  } else if (exp instanceof Expression.Parameter) {
    return exp.path.toString();
  } else if (exp instanceof Expression.FunctionCall) {
    return `${exp.name}(${exp.args.map((a) => stringifyExpression(a)).join(", ")})`;
  } else if (exp instanceof Expression.Conditional) {
    let str = `CASE WHEN ${stringifyExpression(exp.condition)} THEN ${stringifyExpression(exp.then)}`;
    if (
      exp.otherwise instanceof Expression.Literal &&
      exp.otherwise.value == null
    )
      str += " END";
    else if (exp.otherwise instanceof Expression.Conditional)
      str += stringifyExpression(exp.otherwise).slice(4);
    else str += ` ELSE ${stringifyExpression(exp.otherwise)} END`;
    return str;
  }
  throw new Error("Invalid expression");
 }
 export function parseLikePattern(pat: string, esc: string): string[] {
  const chars = pat.split("");
  for (let i = 0; i < chars.length; ++i) {
    const c = chars[i];
    if (c === esc) {
      chars[i] = chars[i + 1] || "";
      chars[i + 1] = "";
    } else if (c === "_") {
      chars[i] = "\\_";
    } else if (c === "%") {
      chars[i] = "\\%";
      while (chars[i + 1] === "%") chars[++i] = "";
    }
  }
  return chars.filter((c) => c);
 }
 export function likePatternToRegExp(pat: string, esc = "", flags = ""): RegExp {
  const convChars = {
    "-": "\\-",
    "/": "\\/",
    "\\": "\\/",
    "^": "\\^",
    $: "\\$",
    "*": "\\*",
    "+": "\\+",
    "?": "\\?",
    ".": "\\.",
    "(": "\\(",
    ")": "\\)",
    "|": "\\|",
    "[": "\\[",
    "]": "\\]",
    "{": "\\{",
    "}": "\\}",
    "\\%": ".*",
    "\\_": ".",
  };
  let chars = parseLikePattern(pat, esc);
  if (!chars.length) return new RegExp("^$", flags);
  chars = chars.map((c) => convChars[c] || c);
  chars[0] = chars[0] === ".*" ? "" : "^" + chars[0];
  const l = chars.length - 1;
  chars[l] = [".*", ""].includes(chars[l]) ? "" : chars[l] + "$";
  return new RegExp(chars.join(""), flags);
 }
@@ -1,53 +0,0 @@
 let cache1 = new Map();
 let cache2 = new Map();
 const keys = new WeakMap();
 function getKey(obj): string {
  if (obj === null) return "null";
  else if (obj === undefined) return "undefined";
  const t = typeof obj;
  if (t === "number" || t === "boolean" || t === "string") return `${t}:${obj}`;
  if (t !== "function" && t !== "object")
    throw new Error(`Cannot memoize ${t} arguments`);
  let k = keys.get(obj);
  if (!k) {
    const rnd = Math.trunc(Math.random() * Number.MAX_SAFE_INTEGER);
    k = `${t}:${rnd.toString(36)}`;
    keys.set(obj, k);
  }
  return k;
 }
 export default function memoize<T extends (...args: any[]) => any>(func: T): T {
  const funcKey = getKey(func);
  return ((...args) => {
    const key = JSON.stringify(args.map(getKey)) + funcKey;
    if (cache1.has(key)) return cache1.get(key);
    let r;
    if (cache2.has(key)) {
      cache1.set(key, (r = cache2.get(key)));
    } else {
      cache1.set(key, (r = func(...args)));
      // Evict rejected promises
      if (r instanceof Promise) {
        r.catch(() => {
          cache1.delete(key);
          cache2.delete(key);
        });
      }
    }
    return r;
  }) as any;
 }
 const interval = setInterval(() => {
  cache2 = cache1;
  cache1 = new Map();
 }, 120000);
 // Don't hold Node.js process
 if (interval.unref) interval.unref();
@@ -1,115 +0,0 @@
 import Path from "./path.ts";
 export default class PathSet {
  private paramSegmentIndex: Map<string, Set<Path>>[] = [];
  private attrSegmentIndex: Map<string, Set<Path>>[] = [];
  private stringIndex: Map<string, Path> = new Map();
  public constructor() {}
  public add(pathStr: string): Path {
    let path: Path = this.get(pathStr);
    if (path) return path;
    path = Path.parse(pathStr);
    if (path.alias) throw new Error("PathSet does not support aliased paths");
    this.stringIndex.set(path.toString(), path);
    while (this.paramSegmentIndex.length < path.paramLength)
      this.paramSegmentIndex.push(new Map());
    while (this.attrSegmentIndex.length < path.attrLength)
      this.attrSegmentIndex.push(new Map());
    for (let i = 0; i < path.length; ++i) {
      const fragment = path.segments[i] as string;
      const fragmentIndex =
        i < path.paramLength
          ? this.paramSegmentIndex[i]
          : this.attrSegmentIndex[i - path.paramLength];
      let fragmentIndexSet = fragmentIndex.get(fragment);
      if (!fragmentIndexSet) {
        fragmentIndexSet = new Set<Path>();
        fragmentIndex.set(fragment, fragmentIndexSet);
      }
      fragmentIndexSet.add(path);
    }
    return path;
  }
  public get(path: string): Path {
    return this.stringIndex.get(path);
  }
  public findCompat(
    path: Path,
    superset = false,
    subset = false,
    depth = path.length,
  ): Path[] {
    if (path.attrLength)
      throw new Error("findCompat() does not support attribute paths");
    depth = Math.min(31, depth);
    const paramMask = (1 << path.paramLength) - 1;
    let mask = ((0b10 << depth) - 1) & ~paramMask;
    if (superset) mask |= ~path.wildcard & paramMask;
    if (subset) mask |= path.wildcard;
    return this.find(path, mask, 1);
  }
  public find(path: Path, paramMask: number, attrMask: number): Path[] {
    if (path.alias) throw new Error("PathSet does not support aliased paths");
    if (path.paramLength > this.paramSegmentIndex.length) return [];
    if (path.attrLength > this.attrSegmentIndex.length) return [];
    const emptySet: Set<Path> = new Set();
    const indexes: [Set<Path>, Set<Path>][] = [];
    const paramLengthMask = (1 << path.paramLength) - 1;
    const attrLengthMask = (1 << path.attrLength) - 1;
    const mask = (paramMask & paramLengthMask) | (attrMask << path.paramLength);
    for (const [i, s] of path.segments.entries()) {
      const b = 1 << i;
      const m = mask & b;
      const w = b & path.wildcard;
      if (w && m) continue;
      const idxSet =
        i < path.paramLength
          ? this.paramSegmentIndex[i]
          : this.attrSegmentIndex[i - path.paramLength];
      const idx1 = idxSet.get(s as string) || emptySet;
      let idx2 = emptySet;
      if (m) idx2 = idxSet.get("*") || emptySet;
      indexes.push([idx1, idx2]);
    }
    indexes.sort((a, b) => a[0].size + a[1].size - (b[0].size + b[1].size));
    let res: Path[];
    if (!indexes.length) res = [...this.stringIndex.values()];
    else res = [...indexes[0][0], ...indexes[0][1]];
    const paramCover = ~paramLengthMask & paramMask;
    const attrCover = ~attrLengthMask & attrMask;
    res = res.filter(
      (p) =>
        (1 << p.paramLength) & paramCover && (1 << p.attrLength) & attrCover,
    );
    for (let i = 1; i < indexes.length; ++i) {
      const [idx1, idx2] = indexes[i];
      res = res.filter((p) => idx1.has(p) || idx2.has(p));
    }
    return res;
  }
 }
@@ -1,179 +0,0 @@
 import Expression from "./expression.ts";
 import { Cursor, parsePath } from "./expression/parser.ts";
 type Segments = (string | Expression)[];
 let cache1 = new Map<string, Path>();
 let cache2 = new Map<string, Path>();
 export default class Path {
  declare public readonly segments: Segments;
  declare public readonly colon: number;
  declare public readonly wildcard: number;
  declare public readonly alias: number;
  declare protected _string: string;
  declare protected _stringIndex: number[];
  constructor(segments: Segments, colon: number) {
    if (!(colon <= segments.length)) throw new Error("Invalid path");
    if (segments.length > 32) throw new Error("Path too long");
    Object.freeze(segments);
    let alias = 0;
    let wildcard = 0;
    const arr = segments.map((s, i) => {
      if (s instanceof Expression) {
        alias |= 1 << i;
        return `[${s.toString()}]`;
      } else if (s === "*") {
        wildcard |= 1 << i;
      }
      return s;
    });
    let offset = 0;
    const stringIndex = arr.map((s, i) => (offset += s.length) + i);
    this.segments = segments;
    this.colon = colon;
    this.wildcard = wildcard;
    this.alias = alias;
    if (!colon) this._string = arr.join(".");
    else
      this._string =
        arr.slice(0, -colon).join(".") + ":" + arr.slice(-colon).join(".");
    this._stringIndex = stringIndex;
  }
  public static parse(input: string): Path {
    let path = cache1.get(input);
    if (!path) {
      path = cache2.get(input);
      if (!path) {
        const cursor = new Cursor(input);
        path = parsePath(cursor);
        if (cursor.charCode) throw new Error("Unexpected character");
        if (path.toString() !== input) cache1.set(path.toString(), path);
      }
      cache1.set(input, path);
    }
    return path;
  }
  public get length(): number {
    return this.segments.length;
  }
  public get paramLength(): number {
    return this.segments.length - this.colon;
  }
  public get attrLength(): number {
    return this.colon;
  }
  public toString(): string {
    return this._string;
  }
  public slice(start = 0, end: number = this.segments.length): Path {
    if (start < 0) start = Math.max(0, this.segments.length + start);
    if (end < 0) end = Math.max(0, this.segments.length + end);
    if (start >= end) return Path.root;
    let i1 = start > 0 ? this._stringIndex[start - 1] + 1 : 0;
    // Include the ":" when slicing exactly at the colon boundary
    if (this.colon && start === this.segments.length - this.colon) --i1;
    const i2 =
      end <= this.segments.length
        ? this._stringIndex[end - 1]
        : this._string.length;
    const str = this._string.slice(i1, i2);
    let path = cache1.get(str);
    if (!path) {
      path = cache2.get(str);
      if (!path) {
        const segments = this.segments.slice(start, end);
        const colon =
          start <= this.segments.length - this.colon
            ? Math.max(0, this.colon - this.segments.length + end)
            : 0;
        path = new Path(segments, colon);
      }
      cache1.set(str, path);
    }
    return path;
  }
  public concat(path2: Path): Path {
    if (!path2._string) return this;
    else if (!this._string) return path2;
    if (this.colon && path2.colon && path2.colon < path2.segments.length)
      throw new Error("Invalid path");
    const colon = this.colon ? this.colon + path2.segments.length : path2.colon;
    let str;
    if (this.colon && path2.colon === path2.segments.length) {
      // Right is all-colon; strip its ":" prefix and join with "."
      str = `${this._string}.${path2._string.slice(1)}`;
    } else if (path2.colon === path2.segments.length) {
      // Left has no colon; right is all-colon; concatenate directly
      str = `${this._string}${path2._string}`;
    } else {
      str = `${this._string}.${path2._string}`;
    }
    let path = cache1.get(str);
    if (!path) {
      path = cache2.get(str);
      if (!path) {
        const segments = this.segments.concat(path2.segments);
        path = new Path(segments, colon);
      }
      cache1.set(str, path);
    }
    return path;
  }
  public stripAlias(): Path {
    if (!this.alias) return this;
    const segments = this.segments.map((s) =>
      s instanceof Expression ? "*" : s,
    );
    let str: string;
    if (!this.colon) str = segments.join(".");
    else
      str =
        segments.slice(0, -this.colon).join(".") +
        ":" +
        segments.slice(-this.colon).join(".");
    let path = cache1.get(str);
    if (!path) {
      path = cache2.get(str);
      if (!path) {
        path = new Path(segments, this.colon);
      }
      cache1.set(str, path);
    }
    return path;
  }
  static root = new Path([], 0);
 }
 const interval = setInterval(() => {
  cache2 = cache1;
  cache1 = new Map();
 }, 120000);
 // Don't hold Node.js process
 if (interval.unref) interval.unref();
@@ -1,218 +0,0 @@
 const LINE_WIDTH = 80;
 const INDENTATION = "  ";
 const STRING_RESERVED = new Set([
  "true",
  "True",
  "TRUE",
  "false",
  "False",
  "FALSE",
  "null",
  "Null",
  "NULL",
 ]);
 function isPrintable(str: string): boolean {
  return !/[^\t\n\x20-\x7e\x85\u{a0}-\u{d7ff}\u{e000}-\u{fffd}\u{10000}-\u{10ffff}]/u.test(
    str,
  );
 }
 function stringifyKey(str: string): string {
  if (!str || !isPrintable(str)) return JSON.stringify(str);
  if (/^[\s-?:,[\]{}#&$!|>'"%@`]|: | #|[\n,[\]{}]|\s$/.test(str))
    return JSON.stringify(str);
  return str;
 }
 function foldString(str: string): string[] {
  if (str.length <= LINE_WIDTH) return [str];
  if (str.startsWith(" ")) return [str];
  const lines: string[] = [];
  let idx = 0;
  let cand = 0;
  for (let i = 1; i < str.length - 1; ++i) {
    if (str[i] !== " ") continue;
    if (str[i + 1] === " ") {
      i += 2;
      while (str[i] === " ") ++i;
      continue;
    }
    if (i <= idx + LINE_WIDTH) {
      cand = i;
      continue;
    }
    const c = cand > idx ? cand : i;
    lines.push(str.slice(idx, c));
    idx = c + 1;
    cand = i;
  }
  if (cand > idx && str.length > idx + LINE_WIDTH) {
    lines.push(str.slice(idx, cand));
    idx = cand + 1;
  }
  lines.push(str.slice(idx));
  return lines;
 }
 function stringifyString(str: string, res: string[], prefix1, prefix2): void {
  if (/^\s*$/.test(str) || STRING_RESERVED.has(str) || !isPrintable(str)) {
    res.push(prefix1 + JSON.stringify(str));
    return;
  }
  if (!prefix2) prefix2 = INDENTATION;
  const lines = str.split("\n");
  if (lines.length > 1) {
    let idt = "";
    let chmp = "-";
    if ((lines.find((l) => l) || "").startsWith(" "))
      idt = `${INDENTATION.length}`;
    if (!lines[lines.length - 1]) {
      lines.pop();
      if (lines[lines.length - 1]) chmp = "";
      else chmp = "+";
    }
    if (/^\s+$/.test(lines[lines.length - 1])) {
      res.push(prefix1 + JSON.stringify(str));
      return;
    }
    let isFolded = false;
    const folded = lines.map((l) => {
      const ls = foldString(l);
      if (ls.length > 1) isFolded = true;
      return ls;
    });
    if (!isFolded) {
      res.push(
        `${prefix1}|${idt}${chmp}`,
        ...lines.map((l) => (l ? prefix2 + l : l)),
      );
      return;
    }
    res.push(`${prefix1}>${idt}${chmp}`);
    res.push(...folded[0].map((f) => prefix2 + f));
    for (let i = 1; i < folded.length; ++i) {
      const prevLine = folded[i - 1][0];
      if (prevLine && !folded[i - 1][0].startsWith(" ")) res.push("");
      res.push(...folded[i].map((f) => prefix2 + f));
    }
    return;
  }
  if (
    /^[\s-?:,[\]{}#&$!|>'"%@`]|: | #|\s$/.test(str) ||
    parseFloat(str) === +str
  ) {
    res.push(prefix1 + JSON.stringify(str));
    return;
  }
  res.push(prefix1 + str);
 }
 function stringifyAny(
  obj: unknown,
  res: string[],
  prefix1 = "",
  prefix2 = "",
 ): void {
  if (obj == null) {
    res.push(`${prefix1}null`);
    return;
  }
  if (typeof obj === "number" || typeof obj === "boolean") {
    res.push(`${prefix1}${JSON.stringify(obj)}`);
    return;
  }
  if (obj instanceof Date) {
    res.push(`${prefix1}${obj.toJSON()}`);
    return;
  }
  if (typeof obj === "string") {
    stringifyString(obj, res, prefix1, prefix2);
    return;
  }
  if (Array.isArray(obj)) {
    if (!obj.length) {
      res.push(prefix1 + "[]");
      return;
    }
    if (!prefix1 || prefix1.endsWith("- ")) {
      stringifyAny(obj[0], res, prefix1 + "- ", prefix2 + INDENTATION);
      prefix1 = prefix2 + "- ";
      prefix2 = prefix2 + INDENTATION;
      for (let i = 1; i < obj.length; ++i)
        stringifyAny(obj[i], res, prefix1, prefix2);
    } else {
      res.push(prefix1);
      prefix1 = prefix2 + "- ";
      prefix2 = prefix2 + INDENTATION;
      for (let i = 0; i < obj.length; ++i)
        stringifyAny(obj[i], res, prefix1, prefix2);
    }
    return;
  }
  const entries = Object.entries(obj).filter((e) => e[1] !== undefined);
  if (!entries.length) {
    res.push(prefix1 + "{}");
    return;
  }
  if (!prefix1 || prefix1.endsWith("- ")) {
    stringifyAny(
      entries[0][1],
      res,
      prefix1 + `${stringifyKey(entries[0][0])}: `,
      prefix2 + INDENTATION,
    );
    prefix1 = prefix2;
    prefix2 = prefix2 + INDENTATION;
    for (let i = 1; i < entries.length; ++i) {
      stringifyAny(
        entries[i][1],
        res,
        prefix1 + `${stringifyKey(entries[i][0])}: `,
        prefix2,
      );
    }
  } else {
    res.push(prefix1);
    prefix1 = prefix2;
    prefix2 = prefix2 + INDENTATION;
    for (let i = 0; i < entries.length; ++i) {
      stringifyAny(
        entries[i][1],
        res,
        prefix1 + `${stringifyKey(entries[i][0])}: `,
        prefix2,
      );
    }
  }
 }
 export function stringify(obj: unknown): string {
  if (obj === undefined) return undefined;
  const lines: string[] = [];
  stringifyAny(obj, lines);
  return lines.join("\n") + "\n";
 }
@@ -0,0 +1,171 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 ###
 path = require 'path'
 common = require './common'
 options = {
  CONFIG_DIR : {type : 'string', default : 'config'},
  MONGODB_CONNECTION_URL : {type : 'string', default : 'mongodb://127.0.0.1/genieacs'},
  REDIS_PORT : {type : 'int', default : 6379},
  REDIS_HOST : {type : 'string', default : '127.0.0.1'},
  REDIS_DB : {type : 'int', default : 0},
  CWMP_WORKER_PROCESSES : {type : 'int', default : 4},
  CWMP_PORT : {type : 'int', default : 7547},
  CWMP_INTERFACE : {type : 'string', default : '0.0.0.0'},
  CWMP_SSL : {type : 'bool', default : false},
  NBI_WORKER_PROCESSES : {type : 'int', default : 2},
  NBI_PORT : {type : 'int', default : 7557},
  NBI_INTERFACE : {type : 'string', default : '0.0.0.0'},
  NBI_SSL : {type : 'bool', default : false},
  FS_WORKER_PROCESSES : {type : 'int', default : 2},
  FS_PORT : {type : 'int', default : 7567},
  FS_INTERFACE : {type : 'string', default : '0.0.0.0'},
  FS_SSL : {type : 'bool', default : false},
  FS_IP : {type : 'string', default : '192.168.0.1'},
  PRESETS_CACHE_DURATION : {type : 'int', default : 86400},
  LOG_INFORMS : {type : 'bool', default : true},
  DEBUG : {type : 'bool', default : false},
  RETRY_DELAY : {type : 'int', default : 300},
  IGNORE_XML_NAMESPACES : {type : 'bool', default : false},
  SESSION_TIMEOUT : {type : 'int', default : 30},
  GET_PARAMETER_NAMES_DEPTH_THRESHOLD : {type : 'int', default : 0},
  TASK_PARAMETERS_BATCH_SIZE : {type : 'int', default : 32},
  COOKIES_PATH : {type : 'string'},
  # Libxml related configuration
  XML_PARSE_RECOVER : {type : 'bool'},
  XML_PARSE_NOENT : {type : 'bool'},
  XML_PARSE_NOBLANKS : {type : 'bool'},
  XML_PARSE_NSCLEAN : {type : 'bool'},
  XML_PARSE_NOCDATA : {type : 'bool'},
  XML_PARSE_IGNORE_ENC : {type : 'bool'},
  # Should probably never be changed
  PRESETS_TIME_PADDING : {type : 'int', default : 1},
  DEVICE_ONLINE_THRESHOLD : {type : 'int', default : 4000}
 }
 allConfig = {}
 setConfig = (name, value, commandLineArgument) ->
  return true if allConfig[name]?
  cast = (val, type) ->
    switch type
      when 'int'
        Number(val)
      when 'bool'
        String(val).trim().toLowerCase() in ['true', 'on', 'yes', '1']
      when 'string'
        String(val)
      else
        null
  _value = null
  for optionName, optionDetails of options
    n = optionName
    if commandLineArgument
      n = n.toLowerCase().replace(/_/g, '-')
    if name == n
      _value = cast(value, optionDetails.type)
      n = optionName
    else if common.startsWith(name, "#{n}-")
      _value = cast(value, optionDetails.type)
      n = "#{optionName}-#{name[optionName.length+1..]}"
    if _value?
      allConfig[n] = _value
      # Save as environmnet variable to pass on to any child processes
      process.env[n] = _value
      return true
  return false
 # Command line arguments
 exports.argv = []
 argv = process.argv[2..]
 while argv.length
  arg = argv.shift()
  if arg[0] == '-'
    v = argv.shift()
    exports.argv[arg] = v
    setConfig(arg[2..], v, true)
  else
    exports.argv.push(arg)
 # Environment variable
 for k, v of process.env
  continue if k.lastIndexOf('GENIEACS_', 0) != 0
  k = k[9..] # remove "GENIEACS_" prefix
  setConfig(k, v)
 # Find config dir
 if exports.argv['--config-dir']?
  allConfig.CONFIG_DIR = exports.argv['--config-dir']
 else if process.env['GENIEACS_CONFIG_DIR']?
  allConfig.CONFIG_DIR = process.env['GENIEACS_CONFIG_DIR']
 else
  allConfig.CONFIG_DIR = options.CONFIG_DIR.default
 # Configuration file
 for k, v of require(path.resolve(allConfig.CONFIG_DIR, 'config'))
  setConfig(k, v)
 # Defaults
 for k, v of options
  setConfig(k, v.default) if v.default?
 get = (option, deviceId) ->
  if deviceId?
    name = "#{option}-#{deviceId}"
    v = allConfig[name]
    return v if v?
    i = name.lastIndexOf('-')
    v = allConfig[name[0...i]]
    return v if v?
    i = name.lastIndexOf('-', i-1)
    v = allConfig[name[0...i]]
    return v if v? or i == -1
  return allConfig[option]
 # load parameter configurations
 exports.PARAMETERS = require(path.resolve(allConfig.CONFIG_DIR, 'parameters'))
 # load authentication scripts
 exports.auth = require(path.resolve(allConfig.CONFIG_DIR, 'auth'))
 exports.CUSTOM_COMMANDS = require(path.resolve(allConfig.CONFIG_DIR, 'custom_commands'))
 exports.get = get
 exports.allConfig = allConfig
@@ -1,279 +0,0 @@
 import { resolve } from "node:path";
 import { readFileSync, existsSync } from "node:fs";
 // Find project root directory
 export let ROOT_DIR = resolve(__dirname, "..");
 while (!existsSync(`${ROOT_DIR}/package.json`)) {
  const d = resolve(ROOT_DIR, "..");
  if (d === ROOT_DIR) {
    ROOT_DIR = process.cwd();
    break;
  }
  ROOT_DIR = d;
 }
 // For compatibility with v1.1
 let configDir, cwmpSsl, nbiSsl, fsSsl, uiSsl, fsHostname;
 const options = {
  EXT_DIR: { type: "path", default: resolve(ROOT_DIR, "config/ext") },
  MONGODB_CONNECTION_URL: {
    type: "string",
    default: "mongodb://127.0.0.1/genieacs",
  },
  CWMP_WORKER_PROCESSES: { type: "int", default: 0 },
  CWMP_PORT: { type: "int", default: 7547 },
  CWMP_INTERFACE: { type: "string", default: "::" },
  CWMP_SSL_CERT: { type: "string", default: "" },
  CWMP_SSL_KEY: { type: "string", default: "" },
  CWMP_LOG_FILE: { type: "path", default: "" },
  CWMP_ACCESS_LOG_FILE: { type: "path", default: "" },
  NBI_WORKER_PROCESSES: { type: "int", default: 0 },
  NBI_PORT: { type: "int", default: 7557 },
  NBI_INTERFACE: { type: "string", default: "::" },
  NBI_SSL_CERT: { type: "string", default: "" },
  NBI_SSL_KEY: { type: "string", default: "" },
  NBI_LOG_FILE: { type: "path", default: "" },
  NBI_ACCESS_LOG_FILE: { type: "path", default: "" },
  FS_WORKER_PROCESSES: { type: "int", default: 0 },
  FS_PORT: { type: "int", default: 7567 },
  FS_INTERFACE: { type: "string", default: "::" },
  FS_SSL_CERT: { type: "string", default: "" },
  FS_SSL_KEY: { type: "string", default: "" },
  FS_URL_PREFIX: { type: "string", default: "" },
  FS_LOG_FILE: { type: "path", default: "" },
  FS_ACCESS_LOG_FILE: { type: "path", default: "" },
  UI_WORKER_PROCESSES: { type: "int", default: 0 },
  UI_PORT: { type: "int", default: 3000 },
  UI_INTERFACE: { type: "string", default: "::" },
  UI_SSL_CERT: { type: "string", default: "" },
  UI_SSL_KEY: { type: "string", default: "" },
  UI_LOG_FILE: { type: "path", default: "" },
  UI_ACCESS_LOG_FILE: { type: "path", default: "" },
  UI_JWT_SECRET: { type: "string", default: "" },
  UDP_CONNECTION_REQUEST_PORT: { type: "int", default: 0 },
  FORWARDED_HEADER: { type: "string", default: "" },
  DOWNLOAD_TIMEOUT: { type: "int", default: 3600 },
  EXT_TIMEOUT: { type: "int", default: 3000 },
  MAX_CACHE_TTL: { type: "int", default: 86400 },
  DEBUG_FILE: { type: "path", default: "" },
  DEBUG_FORMAT: { type: "string", default: "yaml" },
  DEBUG: { type: "bool", default: false },
  RETRY_DELAY: { type: "int", default: 300 },
  SESSION_TIMEOUT: { type: "int", default: 30 },
  CONNECTION_REQUEST_TIMEOUT: { type: "int", default: 2000 },
  GPN_NEXT_LEVEL: { type: "int", default: 0 },
  GPV_BATCH_SIZE: { type: "int", default: 32 },
  MAX_DEPTH: { type: "int", default: 16 },
  COOKIES_PATH: { type: "string" },
  LOG_FORMAT: { type: "string", default: "simple" },
  ACCESS_LOG_FORMAT: { type: "string", default: "" },
  MAX_CONCURRENT_REQUESTS: { type: "int", default: 20 },
  DATETIME_MILLISECONDS: { type: "bool", default: true },
  BOOLEAN_LITERAL: { type: "bool", default: true },
  CONNECTION_REQUEST_ALLOW_BASIC_AUTH: { type: "bool", default: false },
  MAX_COMMIT_ITERATIONS: { type: "int", default: 32 },
  // Should probably never be changed
  DEVICE_ONLINE_THRESHOLD: { type: "int", default: 4000 },
  XMPP_JID: { type: "string", default: "" },
  XMPP_PASSWORD: { type: "string", default: "" },
 };
 const allConfig: { [name: string]: string | number } = {};
 function setConfig(name, value, commandLineArgument = false): boolean {
  if (allConfig[name] != null) return true;
  // For compatibility with v1.1
  if (name === "CONFIG_DIR" || name === "config-dir")
    configDir = configDir || resolve(ROOT_DIR, value);
  if (name === "CWMP_SSL" || name === "cwmp-ssl")
    cwmpSsl = cwmpSsl || String(value).toLowerCase().trim();
  if (name === "NBI_SSL" || name === "nbi-ssl")
    nbiSsl = nbiSsl || String(value).toLowerCase().trim();
  if (name === "FS_SSL" || name === "fs-ssl")
    fsSsl = fsSsl || String(value).toLowerCase().trim();
  if (name === "UI_SSL" || name === "ui-ssl")
    uiSsl = uiSsl || String(value).toLowerCase().trim();
  if (name === "FS_HOSTNAME" || name === "fs-hostname")
    fsHostname = fsHostname || String(value).trim();
  // For compatibility with v1.0
  if (name === "PRESETS_CACHE_DURATION" || name === "presets-cache-duration")
    setConfig("MAX_CACHE_TTL", value);
  if (
    name === "GET_PARAMETER_NAMES_DEPTH_THRESHOLD" ||
    name === "get-parameter-names-depth-threshold"
  )
    setConfig("GPN_NEXT_LEVEL", value);
  if (
    name === "TASK_PARAMETERS_BATCH_SIZE" ||
    name === "task-parameters-batch-size"
  )
    setConfig("GPV_BATCH_SIZE", value);
  if (name === "FS_IP" || name === "fs-ip") setConfig("FS_HOSTNAME", value);
  function cast(val, type): string | number | boolean {
    switch (type) {
      case "int":
        return Number(val);
      case "bool":
        return ["true", "1"].includes(String(val).trim().toLowerCase());
      case "string":
        return String(val);
      case "path":
        if (!val) return "";
        return resolve(val);
      default:
        return null;
    }
  }
  let _value = null;
  for (const [optionName, optionDetails] of Object.entries(options)) {
    let n = optionName;
    if (commandLineArgument) n = n.toLowerCase().replace(/_/g, "-");
    if (name === n) {
      _value = cast(value, optionDetails.type);
      n = optionName;
    } else if (name.startsWith(`${n}-`)) {
      _value = cast(value, optionDetails.type);
      n = `${optionName}-${name.slice(optionName.length + 1)}`;
    }
    if (_value != null) {
      allConfig[n] = _value;
      // Save as environmnet variable to pass on to any child process
      process.env[`GENIEACS_${n}`] = _value;
      return true;
    }
  }
  return false;
 }
 // Command line arguments
 const argv = process.argv.slice(2);
 while (argv.length) {
  const arg = argv.shift();
  if (arg[0] === "-") {
    const v = argv.shift();
    setConfig(arg.slice(2), v, true);
  }
 }
 // Environment variable
 for (const [k, v] of Object.entries(process.env))
  if (k.startsWith("GENIEACS_")) setConfig(k.slice(9), v);
 // Configuration file
 const configFilename = configDir
  ? `${configDir}/config.json`
  : `${ROOT_DIR}/config/config.json`;
 if (existsSync(configFilename)) {
  const configFile = JSON.parse(readFileSync(configFilename).toString());
  for (const [k, v] of Object.entries(configFile)) {
    if (!setConfig(k, v))
      // Pass as environment variable to be accessable by extensions
      process.env[`GENIEACS_${k}`] = `${v}`;
  }
 }
 if (configDir) setConfig("EXT_DIR", `${configDir}/ext`);
 if (["true", "1"].includes(cwmpSsl)) {
  const d = configDir || `${ROOT_DIR}/config`;
  setConfig("CWMP_SSL_CERT", `${d}/cwmp.crt`);
  setConfig("CWMP_SSL_KEY", `${d}/cwmp.key`);
 }
 if (["true", "1"].includes(nbiSsl)) {
  const d = configDir || `${ROOT_DIR}/config`;
  setConfig("NBI_SSL_CERT", `${d}/cwmp.crt`);
  setConfig("NBI_SSL_KEY", `${d}/cwmp.key`);
 }
 if (["true", "1"].includes(fsSsl)) {
  const d = configDir || `${ROOT_DIR}/config`;
  setConfig("FS_SSL_CERT", `${d}/cwmp.crt`);
  setConfig("FS_SSL_KEY", `${d}/cwmp.key`);
 }
 if (["true", "1"].includes(uiSsl)) {
  const d = configDir || `${ROOT_DIR}/config`;
  setConfig("UI_SSL_CERT", `${d}/cwmp.crt`);
  setConfig("UI_SSL_KEY", `${d}/cwmp.key`);
 }
 if (fsHostname) {
  const FS_PORT = allConfig["FS_PORT"] || 7567;
  const FS_SSL = !!allConfig["FS_SSL_CERT"];
  setConfig(
    "FS_URL_PREFIX",
    (FS_SSL ? "https" : "http") + `://${fsHostname}:${FS_PORT}/`,
  );
 }
 // Defaults
 for (const [k, v] of Object.entries(options))
  if (v["default"] != null) setConfig(k, v["default"]);
 export function get(
  optionName: string,
  deviceId?: string,
 ): string | number | boolean {
  if (!deviceId) return allConfig[optionName];
  optionName = `${optionName}-${deviceId}`;
  let v = allConfig[optionName];
  if (v != null) return v;
  let i = optionName.lastIndexOf("-");
  v = allConfig[optionName.slice(0, i)];
  if (v != null) return v;
  i = optionName.lastIndexOf("-", i - 1);
  v = allConfig[optionName.slice(0, i)];
  if (v != null) return v;
  i = optionName.lastIndexOf("-", i - 1);
  v = allConfig[optionName.slice(0, i)];
  if (v != null) return v;
  i = optionName.lastIndexOf("-", i - 1);
  if (i > 0) {
    v = allConfig[optionName.slice(0, i)];
    if (v != null) return v;
  }
  return null;
 }
 export function getDefault(optionName: string): string | number | boolean {
  const option = options[optionName];
  if (!option) return null;
  let val = option["default"];
  if (val && option.type === "path") val = resolve(val);
  return val;
 }
@@ -1,307 +0,0 @@
 import * as crypto from "node:crypto";
 import * as dgram from "node:dgram";
 import * as http from "node:http";
 import Expression, { Value } from "./common/expression.ts";
 import * as auth from "./auth.ts";
 import * as extensions from "./extensions.ts";
 import * as debug from "./debug.ts";
 import XmppClient from "./xmpp-client.ts";
 import * as config from "../lib/config.ts";
 import { encodeEntities, parseAttrs, Element } from "./xml-parser.ts";
 import * as logger from "../lib/logger.ts";
 async function extractAuth(
  exp: Expression,
  dflt: Value,
 ): Promise<[string, string, Expression]> {
  let username: string, password: string;
  const _exp = await exp.evaluateAsync(
    async (e: Expression): Promise<Expression> => {
      if (e instanceof Expression.Parameter)
        return new Expression.Literal(null);
      if (e instanceof Expression.FunctionCall) {
        if (e.name === "NOW") return new Expression.Literal(0);
        if (!username) {
          if (e.name === "EXT") {
            if (!e.args.every((a) => a instanceof Expression.Literal))
              return new Expression.Literal(null);
            const args = e.args.map((a) => a.value.toString());
            if (typeof args[0] !== "string" || typeof args[1] !== "string")
              return new Expression.Literal(null);
            const { fault, value } = await extensions.run(args);
            if (fault) return new Expression.Literal(null);
            return new Expression.Literal(value);
          } else if (e.name === "AUTH") {
            if (e.args.every((a) => a instanceof Expression.Literal)) {
              username = `${e.args[0].value ?? ""}`;
              password = `${e.args[1].value ?? ""}`;
            }
            return new Expression.Literal(dflt);
          }
        }
      }
      return e;
    },
  );
  return [username, password, _exp];
 }
 function httpGet(
  url: URL,
  options: http.RequestOptions,
  _debug: boolean,
  deviceId: string,
 ): Promise<{ statusCode: number; headers: http.IncomingHttpHeaders }> {
  return new Promise((resolve, reject) => {
    const req = http
      .get(url, options, (res) => {
        res.resume();
        resolve({ statusCode: res.statusCode, headers: res.headers });
        if (_debug) {
          debug.outgoingHttpRequest(req, deviceId, "GET", url, null);
          debug.incomingHttpResponse(res, deviceId, null);
        }
      })
      .on("error", (err) => {
        req.destroy();
        reject(err);
        if (_debug)
          debug.outgoingHttpRequestError(req, deviceId, "GET", url, err);
      })
      .on("timeout", () => {
        req.destroy();
      });
  });
 }
 export async function httpConnectionRequest(
  address: string,
  authExp: Expression,
  allowBasicAuth: boolean,
  timeout: number,
  _debug: boolean,
  deviceId: string,
 ): Promise<string> {
  const url = new URL(address);
  if (url.protocol !== "http:")
    return "Invalid connection request URL or protocol";
  const options: http.RequestOptions = {
    agent: new http.Agent({ maxSockets: 1, keepAlive: true, timeout: timeout }),
  };
  let authHeader: Record<string, string>;
  let username: string;
  let password: string;
  while (!authHeader || (username != null && password != null)) {
    let opts = options;
    if (authHeader) {
      if (authHeader["method"] === "Basic") {
        if (!allowBasicAuth) return "Basic HTTP authentication not allowed";
        opts = Object.assign(
          {
            headers: {
              Authorization: auth.basic(username || "", password || ""),
            },
          },
          options,
        );
      } else if (authHeader["method"] === "Digest") {
        opts = Object.assign(
          {
            headers: {
              Authorization: auth.solveDigest(
                username,
                password,
                url.pathname + url.search,
                "GET",
                null,
                authHeader,
              ),
            },
          },
          options,
        );
      } else {
        return "Unrecognized auth method";
      }
    }
    let res: { statusCode: number; headers: http.IncomingHttpHeaders };
    try {
      res = await httpGet(url, opts, _debug, deviceId);
    } catch (err) {
      // Workaround for some devices unexpectedly closing the connection
      if (authHeader) {
        try {
          res = await httpGet(url, opts, _debug, deviceId);
        } catch (err) {
          return `Connection request error: ${err.message}`;
        }
      }
      if (err["code"] === "ECONNRESET" || err["code"] === "ECONNREFUSED")
        return "Device is offline";
      return `Connection request error: ${err.message}`;
    }
    if (res.statusCode === 200 || res.statusCode === 204) return "";
    // When a Connection Request is received for the Virtual CWMP Device and the Proxied
    // Device is offline the CPE Proxier MUST respond with an HTTP 503 failure
    if (res.statusCode === 503) return "Device is offline";
    if (res.statusCode === 401 && res.headers["www-authenticate"]) {
      try {
        authHeader = auth.parseWwwAuthenticateHeader(
          res.headers["www-authenticate"],
        );
      } catch {
        return "Connection request error: Error parsing www-authenticate header";
      }
      [username, password, authExp] = await extractAuth(authExp, false);
    } else {
      return `Connection request error: Unexpected status code ${res.statusCode}`;
    }
  }
  return "Connection request error: Incorrect connection request credentials";
 }
 export async function udpConnectionRequest(
  host: string,
  port: number,
  authExp: Expression,
  sourcePort = 0,
  _debug: boolean,
  deviceId: string,
 ): Promise<void> {
  const now = Date.now();
  const client = dgram.createSocket({ type: "udp4", reuseAddr: true });
  // When a device is NAT'ed, the UDP Connection Request must originate from
  // the same address and port used by the STUN server, in order to traverse
  // the firewall. This does require that the Genieacs NBI and STUN server
  // are allowed to bind to the same address and port. The STUN server needs
  // to open its UDP port with the SO_REUSEADDR option, allowing the NBI to
  // also bind to the same port.
  if (sourcePort) client.bind({ port: sourcePort, exclusive: true });
  let username: string;
  let password: string;
  [username, password, authExp] = await extractAuth(authExp, null);
  if (username == null) username = "";
  if (password == null) password = "";
  while (username != null && password != null) {
    const ts = Math.trunc(now / 1000);
    const id = Math.trunc(Math.random() * 4294967295);
    const cn = crypto.randomBytes(8).toString("hex");
    const sig = crypto
      .createHmac("sha1", password)
      .update(`${ts}${id}${username}${cn}`)
      .digest("hex");
    const uri = `http://${host}:${port}?ts=${ts}&id=${id}&un=${username}&cn=${cn}&sig=${sig}`;
    const msg = `GET ${uri} HTTP/1.1\r\nHost: ${host}:${port}\r\n\r\n`;
    const message = Buffer.from(msg);
    for (let i = 0; i < 3; ++i) {
      await new Promise<void>((resolve, reject) => {
        client.send(message, 0, message.length, port, host, (err: Error) => {
          if (err) reject(err);
          else resolve();
          if (_debug) debug.outgoingUdpMessage(host, deviceId, port, msg);
        });
      });
    }
    [username, password, authExp] = await extractAuth(authExp, null);
  }
  client.close();
 }
 const XMPP_JID = config.get("XMPP_JID") as string;
 const XMPP_PASSWORD = config.get("XMPP_PASSWORD") as string;
 const XMPP_RESOURCE = crypto.randomBytes(8).toString("hex");
 let xmppClient: XmppClient;
 function xmppClientOnError(err: Error): void {
  xmppClient = null;
  logger.error({
    message: "XMPP exception",
    exception: err,
    pid: process.pid,
  });
 }
 function xmppClientOnClose(): void {
  xmppClient = null;
 }
 export async function xmppConnectionRequest(
  jid: string,
  authExp: Expression,
  timeout: number,
  _debug: boolean,
  deviceId: string,
 ): Promise<string> {
  if (!xmppClient) {
    const [host, username] = XMPP_JID.split("@").reverse();
    xmppClient = await XmppClient.connect({
      host,
      username,
      resource: XMPP_RESOURCE,
      password: XMPP_PASSWORD,
      timeout: 120000,
    });
    xmppClient.on("error", xmppClientOnError);
    xmppClient.on("close", xmppClientOnClose);
    xmppClient.unref();
  }
  let username: string;
  let password: string;
  [username, password, authExp] = await extractAuth(authExp, null);
  while (username != null && password != null) {
    const msg = `<connectionRequest xmlns="urn:broadband-forum-org:cwmp:xmppConnReq-1-0"><username>${encodeEntities(
      username,
    )}</username><password>${encodeEntities(
      password,
    )}</password></connectionRequest>`;
    let res: Element, rawRes: string, rawReq: string;
    try {
      ({ res, rawRes, rawReq } = await xmppClient.sendIqStanza(
        XMPP_JID,
        jid,
        "get",
        msg,
        timeout,
      ));
    } catch (err) {
      return err.message;
    }
    if (_debug) {
      debug.outgoingXmppStanza(deviceId, rawReq);
      debug.incomingXmppStanza(deviceId, rawRes);
    }
    const attrs = parseAttrs(res.attrs);
    const type = attrs.find((a) => a.name === "type");
    if (type && type.value === "result") return "";
    const error = res.children.find((c) => c.name === "error");
    if (!error || !error.children[0])
      return "Unexpected XMPP connection request response";
    if (error.children[0].name === "service-unavailable")
      return "Device is offline";
    if (error.children[0].name !== "not-authorized")
      return "Unexpected XMPP connection request response";
    [username, password, authExp] = await extractAuth(authExp, null);
  }
  return "Incorrect connection request credentials";
 }
@@ -0,0 +1,52 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 ###
 config = require './config'
 common = require './common'
 exports.execute = (deviceId, command, callback) ->
  args = command.split(/\s+/)
  f = args.shift()
  c = args.shift()
  file = require "../config/custom_commands/#{f}"
  file[c](deviceId, args.join(' '), callback)
 exports.getFileCommands = (filename) ->
  commands = []
  f = require("../config/custom_commands/#{filename}")
  for k,v of f
    commands.push(k)
  return commands
 exports.getDeviceCustomCommands = (deviceId) ->
  commands = {}
  for k,v of config.CUSTOM_COMMANDS
    if eval(v).test(deviceId)
      commands[k] = exports.getFileCommands(k)
  return commands
 exports.getDeviceCustomCommandNames = (deviceId) ->
  commands = []
  for k,v of config.CUSTOM_COMMANDS
    if eval(v).test(deviceId)
      commands.push(k)
  return commands
@@ -0,0 +1,566 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This file incorporates work covered by the following copyright and
 # permission notice:
 #
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 config = require './config'
 common = require './common'
 util = require 'util'
 soap = require './soap'
 tasks = require './tasks'
 db = require './db'
 presets = require './presets'
 mongodb = require 'mongodb'
 apiFunctions = require './api-functions'
 customCommands = require './custom-commands'
 crypto = require 'crypto'
 zlib = require 'zlib'
 updateDevice = require './update-device'
 # Used to reject new TR-069 sessions when under load
 holdUntil = Date.now()
 writeResponse = (currentRequest, res) ->
  if config.get('DEBUG', currentRequest.session.deviceId)
    dump = "# RESPONSE #{new Date(Date.now())}\n" + JSON.stringify(res.headers) + "\n#{res.data}\n\n"
    require('fs').appendFile("./debug/#{currentRequest.session.deviceId}.dump", dump, (err) ->
      throw err if err
    )
  # respond using the same content-encoding as the request
  if currentRequest.httpRequest.headers['content-encoding']? and res.data.length > 0
    switch currentRequest.httpRequest.headers['content-encoding']
      when 'gzip'
        res.headers['Content-Encoding'] = 'gzip'
        compress = zlib.gzip
      when 'deflate'
        res.headers['Content-Encoding'] = 'deflate'
        compress = zlib.deflate
  if compress?
    compress(res.data, (err, data) ->
      res.headers['Content-Length'] = data.length
      currentRequest.httpResponse.writeHead(res.code, res.headers)
      currentRequest.httpResponse.end(data)
    )
  else
    res.headers['Content-Length'] = res.data.length
    currentRequest.httpResponse.writeHead(res.code, res.headers)
    currentRequest.httpResponse.end(res.data)
 endSession = (currentRequest) ->
  db.redisClient.del("session_#{currentRequest.sessionId}", (err, res) ->
    throw err if err
    res = soap.response(null)
    writeResponse(currentRequest, res)
  )
 inform = (currentRequest, cwmpRequest) ->
  # If overloaded, ask CPE to retry in 5 mins
  if Date.now() < holdUntil
    res = {code : 503, headers : {'Retry-After' : 300}, data : ''}
    return writeResponse(currentRequest, res)
  now = new Date(Date.now())
  if config.get('LOG_INFORMS', currentRequest.session.deviceId)
    util.log("#{currentRequest.session.deviceId}: Inform (#{cwmpRequest.methodRequest.event}); retry count #{cwmpRequest.methodRequest.retryCount}")
  parameterNames = (p[0] for p in cwmpRequest.methodRequest.parameterList)
  informHash = crypto.createHash('md5').update(JSON.stringify(parameterNames)).update(JSON.stringify(config.CUSTOM_COMMANDS)).digest('hex')
  deviceUpdates = {parameterValues : cwmpRequest.methodRequest.parameterList, informEvents : cwmpRequest.methodRequest.event}
  # Delete all pending tasks on bootstrap (e.g. factory reset)
  if '0 BOOTSTRAP' in cwmpRequest.methodRequest.event
    lastBootstrap = now
    db.tasksCollection.remove({device : currentRequest.session.deviceId}, (err, removed) ->
      throw err if err
    )
  db.redisClient.get("#{currentRequest.session.deviceId}_inform_hash", (err, oldInformHash) ->
    throw err if err
    updateAndRespond = () ->
      updateDevice.commitUpdates(currentRequest.session.deviceId, deviceUpdates, false, (err) ->
        throw err if err
        db.redisClient.setex("session_#{currentRequest.sessionId}", currentRequest.session.sessionTimeout, JSON.stringify(currentRequest.session), (err) ->
          throw err if err
          res = soap.response({
            id : cwmpRequest.id,
            methodResponse : {type : 'InformResponse'},
            cwmpVersion : cwmpRequest.cwmpVersion
          })
          if !!cookiesPath = config.get('COOKIES_PATH', currentRequest.session.deviceId)
            res.headers['Set-Cookie'] = "session=#{currentRequest.sessionId}; Path=#{cookiesPath}"
          else
            res.headers['Set-Cookie'] = "session=#{currentRequest.sessionId}"
          writeResponse(currentRequest, res)
        )
      )
    if oldInformHash == informHash and not lastBootstrap?
      return updateAndRespond()
    db.redisClient.setex("#{currentRequest.session.deviceId}_inform_hash", config.get('PRESETS_CACHE_DURATION', currentRequest.session.deviceId), informHash, (err, res) ->
      throw err if err
    )
    # populate projection and parameterStructure
    parameterStructure = {}
    projection = {}
    projection._customCommands = 1
    projection._timestamp = 1
    projection._lastBootstrap = 1
    for param in parameterNames
      ref = parameterStructure
      path = ''
      for p in param.split('.')
        ref[p] ?= {_path : path}
        ref = ref[p]
        path += p + '.'
        projection[path + '_timestamp'] = 1
    db.devicesCollection.findOne({'_id' : currentRequest.session.deviceId}, projection, (err, device) ->
      throw err if err
      lastBootstrap ?= device?._lastBootstrap
      _tasks = []
      # For any parameters that aren't in the DB model, issue a refresh task
      traverse = (reference, actual) ->
        for k of reference
          continue if k[0] == '_'
          if not actual[k]?._timestamp? or actual[k]._timestamp < lastBootstrap
            _tasks.push({device: currentRequest.session.deviceId, name : 'refreshObject', objectName : reference[k]._path, timestamp : now})
          else
            traverse(reference[k], actual[k])
      if device?
        traverse(parameterStructure, device)
        # Update custom commands if needed
        deviceCustomCommands = {}
        deviceCustomCommands[k] = v._timestamp for k,v of device._customCommands
        for cmd in customCommands.getDeviceCustomCommandNames(currentRequest.session.deviceId)
          if not (deviceCustomCommands[cmd]?._timestamp < lastBootstrap)
            # Increment timestamp by one millisecond to ensure it runs after any refresh task
            _tasks.push({device: currentRequest.session.deviceId, name: 'customCommand', command: "#{cmd} init", timestamp: new Date(now.getTime() + 1)})
          delete deviceCustomCommands[cmd]
        for cmd of deviceCustomCommands
          deviceUpdates.deletedObjects ?= []
          deviceUpdates.deletedObjects.push("_customCommands.#{cmd}")
        # clear presets hash if parameters are potentially modified
        if _tasks.length > 0
          db.redisClient.del("#{currentRequest.session.deviceId}_presets_hash", (err) ->
            throw err if err
          )
        apiFunctions.insertTasks(_tasks, {}, () ->
          updateAndRespond()
        )
      else
        deviceIdDetails = {}
        for k, v of cwmpRequest.methodRequest.deviceId
          deviceIdDetails["_#{k}"] = v
        db.devicesCollection.insert({_id : currentRequest.session.deviceId, _registered : now, _deviceId : deviceIdDetails}, (err) ->
          throw err if err
          util.log("#{currentRequest.session.deviceId}: New device registered")
          _tasks.push({device: currentRequest.session.deviceId, name : 'refreshObject', objectName : '', timestamp : now})
          for cmd in customCommands.getDeviceCustomCommandNames(currentRequest.session.deviceId)
            # Increment timestamp by one millisecond to ensure it runs after any refresh task
            _tasks.push({device: currentRequest.session.deviceId, name: 'customCommand', command: "#{cmd} init", timestamp: new Date(now.getTime() + 1)})
          apiFunctions.insertTasks(_tasks, {}, () ->
            updateAndRespond()
          )
        )
    )
  )
 runTask = (currentRequest, task, methodResponse) ->
  timeDiff = process.hrtime()
  tasks.task(task, methodResponse, (err, status, methodRequest, deviceUpdates) ->
    throw err if err
    timeDiff = process.hrtime(timeDiff)[0] + 1
    if timeDiff > 3 # in seconds
      # Server under load. Hold new sessions temporarily.
      holdUntil = Math.max(Date.now() + timeDiff * 2000, holdUntil)
    save = status & tasks.STATUS_SAVE
    switch status & ~tasks.STATUS_SAVE
      when tasks.STATUS_OK
        f = () ->
          db.redisClient.setex(String(task._id), currentRequest.session.sessionTimeout, JSON.stringify(task), (err) ->
            throw err if err
            res = soap.response({
              id : task._id,
              methodRequest : methodRequest,
              cwmpVersion : currentRequest.session.cwmpVersion
            })
            writeResponse(currentRequest, res)
            updateSessionExpiry(currentRequest)
          )
        if save
          updateDevice.commitUpdates(currentRequest.session.deviceId, deviceUpdates, true, (err) ->
            throw err if err
            db.tasksCollection.update({_id : mongodb.ObjectID(String(task._id))}, {$set : {session : task.session}}, (err) ->
              throw err if err
              f()
            )
          )
        else
          updateDevice.queueUpdates(currentRequest.session.deviceId, deviceUpdates, (err) ->
            f()
          )
      when tasks.STATUS_COMPLETED
        util.log("#{currentRequest.session.deviceId}: Completed task #{task.name}(#{task._id})")
        updateDevice.commitUpdates(currentRequest.session.deviceId, deviceUpdates, true, (err) ->
          throw err if err
          db.tasksCollection.remove({'_id' : mongodb.ObjectID(String(task._id))}, (err, removed) ->
            throw err if err
            db.redisClient.del(String(task._id), (err, res) ->
              throw err if err
              nextTask(currentRequest)
            )
          )
        )
      when tasks.STATUS_FAULT
        retryAfter = config.get('RETRY_DELAY', currentRequest.session.deviceId) * Math.pow(2, task.retries ? 0)
        util.log("#{currentRequest.session.deviceId}: Fault response for task #{task._id}. Retrying after #{retryAfter} seconds.")
        taskUpdate = {fault : task.fault, timestamp : new Date(Date.now() + retryAfter * 1000)}
        if save
          taskUpdate.session = task.session
        db.tasksCollection.update({_id : mongodb.ObjectID(String(task._id))}, {$set : taskUpdate, $inc : {retries : 1}}, (err) ->
          throw err if err
          nextTask(currentRequest)
        )
      else
        throw new Error('Unknown task status')
  )
 isTaskExpired = (task) ->
  task.expiry <= new Date()
 assertPresets = (currentRequest) ->
  db.redisClient.mget("#{currentRequest.session.deviceId}_presets_hash", 'presets_hash', (err, res) ->
    throw err if err
    devicePresetsHash = res[0]
    presetsHash = res[1]
    if devicePresetsHash? and devicePresetsHash == presetsHash
      # no discrepancy, end the session
      endSession(currentRequest)
    else
      PRESETS_CACHE_DURATION = config.get('PRESETS_CACHE_DURATION', currentRequest.session.deviceId)
      db.getPresetsObjectsAliases((allPresets, allObjects, allAliases) ->
        if not presetsHash?
          presetsHash = presets.calculatePresetsHash(allPresets, allObjects)
          db.redisClient.setex('presets_hash', PRESETS_CACHE_DURATION, presetsHash, (err, res) ->
            throw err if err
          )
        presets.getDevicePreset(currentRequest.session.deviceId, allPresets, allObjects, allAliases, (devicePreset) ->
          presets.processDevicePreset(currentRequest.session.deviceId, devicePreset, (taskList, addTags, deleteTags, expiry) ->
            db.redisClient.setex("#{currentRequest.session.deviceId}_presets_hash", Math.floor(Math.max(1, expiry - config.get('PRESETS_TIME_PADDING'))), presetsHash, (err, res) ->
              throw err if err
            )
            if addTags.length + deleteTags.length + taskList.length
              util.log("#{currentRequest.session.deviceId}: Presets discrepancy found")
            if addTags.length + deleteTags.length
              util.log("#{currentRequest.session.deviceId}: Updating tags +(#{addTags}) -(#{deleteTags})")
            if deleteTags.length
              db.devicesCollection.update({'_id' : currentRequest.session.deviceId}, {'$pull' : {'_tags' : {'$in' : deleteTags}}}, {}, (err, count) ->
                throw err if err
              )
            if addTags.length
              db.devicesCollection.update({'_id' : currentRequest.session.deviceId}, {'$addToSet' : {'_tags' : {'$each' : addTags}}}, {}, (err, count) ->
                throw err if err
              )
            if taskList.length
              t.expiry = expiry for t in taskList
              apiFunctions.insertTasks(taskList, allAliases, (err, taskList) ->
                throw err if err
                task = taskList[0]
                util.log("#{currentRequest.session.deviceId}: Started task #{task.name}(#{task._id})")
                runTask(currentRequest, task, {})
              )
            else
              endSession(currentRequest)
          )
        )
      )
  )
 nextTask = (currentRequest) ->
  now = new Date()
  cur = db.tasksCollection.find({'device' : currentRequest.session.deviceId, timestamp : {$lte : now}}).sort(['timestamp']).limit(1)
  cur.nextObject((err, task) ->
    throw err if err
    if not task
      # no more tasks, check presets discrepancy
      assertPresets(currentRequest)
    else if isTaskExpired(task)
      util.log("#{currentRequest.session.deviceId}: Task is expired #{task.name}(#{task._id})")
      db.tasksCollection.remove({'_id' : mongodb.ObjectID(String(task._id))}, {safe: true}, (err, removed) ->
        throw err if err
        nextTask(currentRequest)
      )
    else
      util.log("#{currentRequest.session.deviceId}: Started task #{task.name}(#{task._id})")
      updateDevice.clearUpdatesQueue(currentRequest.session.deviceId, (err) ->
        throw err if err
        runTask(currentRequest, task, {})
      )
  )
 getSession = (httpRequest, callback) ->
  # Separation by comma is important as some devices don't comform to standard
  COOKIE_REGEX = /\s*([a-zA-Z0-9\-_]+?)\s*=\s*"?([a-zA-Z0-9\-_]*?)"?\s*(,|;|$)/g
  while match = COOKIE_REGEX.exec(httpRequest.headers.cookie)
    sessionId = match[2] if match[1] == 'session'
  return callback() if not sessionId?
  db.redisClient.get("session_#{sessionId}", (err, res) ->
    throw err if err
    return callback(sessionId, JSON.parse(res))
  )
 updateSessionExpiry = (currentRequest) ->
  db.redisClient.expire("session_#{currentRequest.sessionId}", currentRequest.session.sessionTimeout, (err, res) ->
    throw err if err
    if not res
      # Resave session in the very rare case when session cache expires while processing request
      db.redisClient.setex("session_#{currentRequest.sessionId}", currentRequest.session.sessionTimeout, JSON.stringify(currentRequest.session), (err) ->
        throw err if err
      )
  )
 listener = (httpRequest, httpResponse) ->
  if httpRequest.method != 'POST'
    httpResponse.writeHead 405, {'Allow': 'POST'}
    httpResponse.end('405 Method Not Allowed')
    return
  if httpRequest.headers['content-encoding']?
    switch httpRequest.headers['content-encoding']
      when 'gzip'
        stream = httpRequest.pipe(zlib.createGunzip())
      when 'deflate'
        stream = httpRequest.pipe(zlib.createInflate())
      else
        httpResponse.writeHead(415)
        httpResponse.end('415 Unsupported Media Type')
        return
  else
    stream = httpRequest
  chunks = []
  bytes = 0
  stream.on('data', (chunk) ->
    chunks.push(chunk)
    bytes += chunk.length
  )
  httpRequest.getBody = () ->
    # Write all chunks into a Buffer
    body = new Buffer(bytes)
    offset = 0
    chunks.forEach((chunk) ->
      chunk.copy(body, offset, 0, chunk.length)
      offset += chunk.length
    )
    return body
  stream.on('end', () ->
    getSession(httpRequest, (sessionId, session) ->
      cwmpRequest = soap.request(httpRequest, session?.cwmpVersion)
      if not session?
        if cwmpRequest.methodRequest?.type isnt 'Inform'
          httpResponse.writeHead(400)
          httpResponse.end('Session is expired')
          return
        deviceId = common.generateDeviceId(cwmpRequest.methodRequest.deviceId)
        sessionId = crypto.randomBytes(8).toString('hex')
        session = {
          deviceId : deviceId,
          cwmpVersion : cwmpRequest.cwmpVersion,
          sessionTimeout : cwmpRequest.sessionTimeout ? config.get('SESSION_TIMEOUT', deviceId)
        }
        httpRequest.connection.setTimeout(session.sessionTimeout * 1000)
      currentRequest = {
        httpRequest : httpRequest,
        httpResponse : httpResponse,
        sessionId : sessionId,
        session : session
      }
      if config.get('DEBUG', currentRequest.session.deviceId)
        dump = "# REQUEST #{new Date(Date.now())}\n" + JSON.stringify(httpRequest.headers) + "\n#{httpRequest.getBody()}\n\n"
        require('fs').appendFile("./debug/#{currentRequest.session.deviceId}.dump", dump, (err) ->
          throw err if err
        )
      if cwmpRequest.methodRequest?
        if cwmpRequest.methodRequest.type is 'Inform'
          inform(currentRequest, cwmpRequest)
        else if cwmpRequest.methodRequest.type is 'GetRPCMethods'
          util.log("#{currentRequest.session.deviceId}: GetRPCMethods")
          res = soap.response({
            id : cwmpRequest.id,
            methodResponse : {type : 'GetRPCMethodsResponse', methodList : ['Inform', 'GetRPCMethods', 'TransferComplete', 'RequestDownload']},
            cwmpVersion : currentRequest.session.cwmpVersion
          })
          writeResponse(currentRequest, res)
          updateSessionExpiry(currentRequest)
        else if cwmpRequest.methodRequest.type is 'TransferComplete'
          # do nothing
          util.log("#{currentRequest.session.deviceId}: Transfer complete")
          res = soap.response({
            id : cwmpRequest.id,
            methodResponse : {type : 'TransferCompleteResponse'},
            cwmpVersion : currentRequest.session.cwmpVersion
          })
          writeResponse(currentRequest, res)
          updateSessionExpiry(currentRequest)
        else if cwmpRequest.methodRequest.type is 'RequestDownload'
          requestDownloadResponse = () ->
            res = soap.response({
              id : cwmpRequest.id,
              methodResponse : {type : 'RequestDownloadResponse'},
              cwmpVersion : currentRequest.session.cwmpVersion
            })
            writeResponse(currentRequest, res)
            updateSessionExpiry(currentRequest)
          fileType = cwmpRequest.methodRequest.fileType
          util.log("#{currentRequest.session.deviceId}: RequestDownload (#{fileType})")
          if fileType isnt '1 Firmware Upgrade Image'
            # Only supporting firmware upgrade for now
            return requestDownloadResponse()
          db.getPresetsObjectsAliases((allPresets, allObjects, allAliases) ->
            presets.getDevicePreset(currentRequest.session.deviceId, allPresets, allObjects, allAliases, (devicePreset) ->
              presetSoftwareVersion = devicePreset.softwareVersion?.preset
              currentSoftwareVersion = devicePreset.softwareVersion?.current._value
              if presetSoftwareVersion? and presetSoftwareVersion != currentSoftwareVersion
                deviceIdStruct = common.parseDeviceId(currentRequest.session.deviceId)
                db.filesCollection.findOne({'metadata.fileType' : '1 Firmware Upgrade Image', 'metadata.oui' : deviceIdStruct.oui, 'metadata.productClass' : deviceIdStruct.productClass, 'metadata.version' : presetSoftwareVersion}, {_id : 1}, (err, file) ->
                  throw err if err
                  if not file?
                    util.error("#{currentRequest.session.deviceId}: Firmware image not found (#{presetSoftwareVersion})")
                    return requestDownloadResponse()
                  task = {
                    device : currentRequest.session.deviceId,
                    name : 'download',
                    file : file['_id']
                  }
                  apiFunctions.insertTasks(task, allAliases, (err, tasks) ->
                    throw err if err
                    return requestDownloadResponse()
                  )
                )
              else
                return requestDownloadResponse()
            )
          )
        else
          throw new Error('ACS method not supported')
      else if cwmpRequest.methodResponse?
        taskId = cwmpRequest.id
        db.getTask(taskId, (err, task) ->
          throw err if err
          if not task
            nextTask(currentRequest)
          else
            runTask(currentRequest, task, cwmpRequest.methodResponse)
        )
      else if cwmpRequest.fault?
        taskId = cwmpRequest.id
        if not taskId
          # Fault not related to a task. End the session.
          return endSession(currentRequest)
        db.getTask(taskId, (err, task) ->
          throw err if err
          if not task
            nextTask(currentRequest)
          else
            runTask(currentRequest, task, cwmpRequest.fault)
        )
      else
        # cpe sent empty response. start sending acs requests
        nextTask(currentRequest)
    )
  )
 exports.listener = listener
@@ -1,727 +0,0 @@
 import { ObjectId } from "mongodb";
 import { decodeTag, encodeTag, escapeRegExp } from "../util.ts";
 import {
  DeviceData,
  Attributes,
  SessionFault,
  Task,
  Operation,
 } from "../types.ts";
 import { collections } from "../db/db.ts";
 import { optimizeProjection } from "../db/util.ts";
 import * as MongoTypes from "../db/types.ts";
 const INVALID_PATH_SUFFIX = "__invalid";
 function compareAccessLists(list1: string[], list2: string[]): boolean {
  if (list1.length !== list2.length) return false;
  for (const [i, v] of list1.entries()) if (v !== list2[i]) return false;
  return true;
 }
 export async function fetchDevice(
  id: string,
  timestamp: number,
 ): Promise<[string, number, Attributes?][]> {
  const res: [string, number, Attributes?][] = [
    ["Events", timestamp, { object: [timestamp, 1], writable: [timestamp, 0] }],
    [
      "DeviceID",
      timestamp,
      { object: [timestamp, 1], writable: [timestamp, 0] },
    ],
  ];
  const device = (await collections.devices.findOne({ _id: id })) as any;
  if (!device) return null;
  function storeParams(
    obj,
    path: string,
    pathLength: number,
    ts: number,
  ): void {
    if (obj["_timestamp"]) obj["_timestamp"] = +obj["_timestamp"];
    if (obj["_attributesTimestamp"])
      obj["_attributesTimestamp"] = +obj["_attributesTimestamp"];
    const attrs: Attributes = {};
    let t = obj["_timestamp"] || 1;
    if (ts > t) t = ts;
    if (obj["_value"] != null) {
      attrs.value = [obj["_timestamp"] || 1, [obj["_value"], obj["_type"]]];
      if (obj["_type"] === "xsd:dateTime" && obj["_value"] instanceof Date)
        attrs.value[1][0] = +attrs.value[1][0];
      obj["_object"] = false;
    }
    if (obj["_writable"] != null)
      attrs.writable = [ts || 1, obj["_writable"] ? 1 : 0];
    if (obj["_object"] != null) attrs.object = [t, obj["_object"] ? 1 : 0];
    if (obj["_notification"] != null) {
      attrs.notification = [
        obj["_attributesTimestamp"] || 1,
        obj["_notification"],
      ];
    }
    if (obj["_accessList"] != null)
      attrs.accessList = [obj["_attributesTimestamp"] || 1, obj["_accessList"]];
    try {
      res.push([path, t, attrs]);
    } catch {
      // The path parser is now more strict so we might be in a situation where
      // the database contains invalid paths from before this change So here we
      // encode the invalid characters.
      const splits = path.split(".");
      splits[splits.length - 1] =
        encodeTag(splits[splits.length - 1]) + INVALID_PATH_SUFFIX;
      path = splits.join(".");
      res.push([path, t, attrs]);
      return;
    }
    for (const [k, v] of Object.entries(obj)) {
      if (!k.startsWith("_")) {
        obj["_object"] = true;
        storeParams(v, `${path}.${k}`, pathLength + 1, obj["_timestamp"]);
      }
    }
    if (obj["_object"] && obj["_timestamp"])
      res.push([path + ".*", obj["_timestamp"]]);
  }
  const ts: number = +device["_timestamp"] || 0;
  if (ts) res.push(["*", ts]);
  for (const [k, v] of Object.entries(device)) {
    switch (k) {
      case "_lastInform":
        res.push([
          "Events.Inform",
          +v,
          {
            object: [+v, 0],
            writable: [+v, 0],
            value: [+v, [+v, "xsd:dateTime"]],
          },
        ]);
        break;
      case "_lastBoot":
        res.push([
          "Events.1_BOOT",
          +v,
          {
            object: [+v, 0],
            writable: [+v, 0],
            value: [+v, [+v, "xsd:dateTime"]],
          },
        ]);
        break;
      case "_lastBootstrap":
        res.push([
          "Events.0_BOOTSTRAP",
          +v,
          {
            object: [+v, 0],
            writable: [+v, 0],
            value: [+v, [+v, "xsd:dateTime"]],
          },
        ]);
        break;
      case "_registered":
        // Use current timestamp for registered event attribute timestamps
        res.push([
          "Events.Registered",
          timestamp,
          {
            object: [timestamp, 0],
            writable: [timestamp, 0],
            value: [timestamp, [+v, "xsd:dateTime"]],
          },
        ]);
        break;
      case "_id":
        res.push([
          "DeviceID.ID",
          timestamp,
          {
            object: [timestamp, 0],
            writable: [timestamp, 0],
            value: [timestamp, [v as string, "xsd:string"]],
          },
        ]);
        break;
      case "_tags":
        if ((v as string[]).length) {
          res.push([
            "Tags",
            timestamp,
            { object: [timestamp, 1], writable: [timestamp, 0] },
          ]);
        }
        for (const t of v as string[]) {
          res.push([
            "Tags." + encodeTag(t),
            timestamp,
            {
              object: [timestamp, 0],
              writable: [timestamp, 1],
              value: [timestamp, [true, "xsd:boolean"]],
            },
          ]);
        }
        break;
      case "_deviceId":
        if (v["_Manufacturer"] != null) {
          res.push([
            "DeviceID.Manufacturer",
            timestamp,
            {
              object: [timestamp, 0],
              writable: [timestamp, 0],
              value: [timestamp, [v["_Manufacturer"], "xsd:string"]],
            },
          ]);
        }
        if (v["_OUI"] != null) {
          res.push([
            "DeviceID.OUI",
            timestamp,
            {
              object: [timestamp, 0],
              writable: [timestamp, 0],
              value: [timestamp, [v["_OUI"], "xsd:string"]],
            },
          ]);
        }
        if (v["_ProductClass"] != null) {
          res.push([
            "DeviceID.ProductClass",
            timestamp,
            {
              object: [timestamp, 0],
              writable: [timestamp, 0],
              value: [timestamp, [v["_ProductClass"], "xsd:string"]],
            },
          ]);
        }
        if (v["_SerialNumber"] != null) {
          res.push([
            "DeviceID.SerialNumber",
            timestamp,
            {
              object: [timestamp, 0],
              writable: [timestamp, 0],
              value: [timestamp, [v["_SerialNumber"], "xsd:string"]],
            },
          ]);
        }
        break;
      default:
        if (!k.startsWith("_")) storeParams(v, k, 1, ts);
    }
  }
  return res;
 }
 export async function saveDevice(
  deviceId: string,
  deviceData: DeviceData,
  isNew: boolean,
  sessionTimestamp: number,
 ): Promise<void> {
  const update = { $set: {}, $unset: {}, $addToSet: {}, $pull: {} };
  for (const diff of deviceData.timestamps.diff()) {
    if (diff[0].wildcard !== 1 << (diff[0].length - 1)) continue;
    if (
      diff[0].segments[0] === "Events" ||
      diff[0].segments[0] === "DeviceID" ||
      diff[0].segments[0] === "Tags"
    )
      continue;
    const parent = deviceData.paths.get(diff[0].slice(0, -1).toString());
    // Param timestamps may be greater than session timestamp to track revisions
    if (diff[2] > sessionTimestamp) diff[2] = sessionTimestamp;
    if (diff[2] == null && diff[1] != null) {
      update["$unset"][
        parent.length ? parent.toString() + "._timestamp" : "_timestamp"
      ] = 1;
    } else {
      if (parent && (!parent.length || deviceData.attributes.has(parent))) {
        update["$set"][
          parent.length ? parent.toString() + "._timestamp" : "_timestamp"
        ] = new Date(diff[2]);
      }
    }
  }
  for (const diff of deviceData.attributes.diff()) {
    const path = diff[0];
    const value1 = (((diff[1] || {}).value || [])[1] || [])[0];
    const value2 = (((diff[2] || {}).value || [])[1] || [])[0];
    const valueType1 = (((diff[1] || {}).value || [])[1] || [])[1];
    const valueType2 = (((diff[2] || {}).value || [])[1] || [])[1];
    const valueTimestamp1 = ((diff[1] || {}).value || [])[0];
    const valueTimestamp2 = ((diff[2] || {}).value || [])[0];
    const object1 = ((diff[1] || {}).object || [])[1];
    const object2 = ((diff[2] || {}).object || [])[1];
    const writable2 = ((diff[2] || {}).writable || [])[1];
    const writable1 = ((diff[1] || {}).writable || [])[1];
    const attributesTimestamp1 = ((diff[1] || {}).notification || [])[0];
    const attributesTimestamp2 = ((diff[2] || {}).notification || [])[0];
    const notification1 = ((diff[1] || {}).notification || [])[1];
    const notification2 = ((diff[2] || {}).notification || [])[1];
    const accessList1 = ((diff[1] || {}).accessList || [])[1];
    const accessList2 = ((diff[2] || {}).accessList || [])[1];
    switch (path.segments[0]) {
      case "Events":
        if (path.length === 2 && value2 !== value1) {
          if (!diff[2]) {
            switch (path.segments[1]) {
              case "Inform":
                update["$unset"]["_lastInform"] = 1;
                break;
              case "1_BOOT":
                update["$unset"]["_lastBoot"] = 1;
                break;
              case "0_BOOTSTRAP":
                update["$unset"]["_lastBootstrap"] = 1;
                break;
              case "Registered":
                update["$unset"]["_registered"] = 1;
            }
          } else {
            const t = new Date(diff[2].value[1][0] as number);
            switch (path.segments[1]) {
              case "Inform":
                update["$set"]["_lastInform"] = t;
                break;
              case "1_BOOT":
                update["$set"]["_lastBoot"] = t;
                break;
              case "0_BOOTSTRAP":
                update["$set"]["_lastBootstrap"] = t;
                break;
              case "Registered":
                update["$set"]["_registered"] = t;
            }
          }
        }
        break;
      case "DeviceID":
        if (value2 !== value1) {
          const v = diff[2].value[1][0];
          switch (path.segments[1]) {
            case "ID":
              update["$set"]["_id"] = v;
              break;
            case "Manufacturer":
              update["$set"]["_deviceId._Manufacturer"] = v;
              break;
            case "OUI":
              update["$set"]["_deviceId._OUI"] = v;
              break;
            case "ProductClass":
              update["$set"]["_deviceId._ProductClass"] = v;
              break;
            case "SerialNumber":
              update["$set"]["_deviceId._SerialNumber"] = v;
          }
        }
        break;
      case "Tags":
        if (value2 !== value1) {
          if (value2 != null) {
            if (!update["$addToSet"]["_tags"])
              update["$addToSet"]["_tags"] = { $each: [] };
            update["$addToSet"]["_tags"]["$each"].push(
              decodeTag(path.segments[1] as string),
            );
          } else {
            if (!update["$pull"]["_tags"]) {
              update["$pull"]["_tags"] = {
                $in: [],
              };
            }
            update["$pull"]["_tags"]["$in"].push(
              decodeTag(path.segments[1] as string),
            );
          }
        }
        break;
      default:
        if (!diff[2]) {
          let pathStr = path.toString();
          // Paths with that suffix are encoded and need to be decoded
          if (pathStr.endsWith(INVALID_PATH_SUFFIX)) {
            const splits = pathStr.split(".");
            splits[splits.length - 1] = decodeTag(
              splits[splits.length - 1].slice(
                0,
                0 - INVALID_PATH_SUFFIX.length,
              ),
            );
            pathStr = splits.join(".");
          }
          update["$unset"][pathStr] = 1;
          continue;
        }
        for (const attrName of Object.keys(diff[2])) {
          // Param timestamps may be greater than session timestamp to track revisions
          if (diff[2][attrName][0] > sessionTimestamp)
            diff[2][attrName][0] = sessionTimestamp;
          if (diff[2][attrName][1] != null) {
            switch (attrName) {
              case "value":
                if (value2 !== value1) {
                  if (
                    valueType2 === "xsd:dateTime" &&
                    Number.isInteger(value2 as number)
                  ) {
                    update["$set"][path.toString() + "._value"] = new Date(
                      value2 as number,
                    );
                  } else {
                    update["$set"][path.toString() + "._value"] = value2;
                  }
                }
                if (valueType2 !== valueType1)
                  update["$set"][path.toString() + "._type"] = valueType2;
                if (valueTimestamp2 !== valueTimestamp1) {
                  update["$set"][path.toString() + "._timestamp"] = new Date(
                    valueTimestamp2,
                  );
                }
                break;
              case "object":
                if (!diff[1]?.object || object2 !== object1) {
                  update["$set"][
                    path.length ? path.toString() + "._object" : "_object"
                  ] = !!object2;
                }
                break;
              case "writable":
                if (!diff[1]?.writable || writable2 !== writable1) {
                  update["$set"][
                    path.length ? path.toString() + "._writable" : "_writable"
                  ] = !!writable2;
                }
                break;
              case "notification":
                if (
                  !diff[1] ||
                  !diff[1].notification ||
                  notification2 !== notification1
                ) {
                  update["$set"][
                    path.length
                      ? path.toString() + "._notification"
                      : "_notification"
                  ] = notification2;
                }
                if (attributesTimestamp2 !== attributesTimestamp1) {
                  update["$set"][path.toString() + "._attributesTimestamp"] =
                    new Date(attributesTimestamp2);
                }
                break;
              case "accessList":
                if (
                  !diff[1] ||
                  !diff[1].accessList ||
                  !compareAccessLists(accessList2, accessList1)
                ) {
                  update["$set"][
                    path.length
                      ? path.toString() + "._accessList"
                      : "_accessList"
                  ] = accessList2;
                }
                if (attributesTimestamp2 !== attributesTimestamp1) {
                  update["$set"][path.toString() + "._attributesTimestamp"] =
                    new Date(attributesTimestamp2);
                }
            }
          }
        }
        if (diff[1]) {
          for (const attrName of Object.keys(diff[1])) {
            if (
              diff[1][attrName][1] != null &&
              diff[2]?.[attrName]?.[1] == null
            ) {
              const p = path.length ? path.toString() + "." : "";
              update["$unset"][`${p}_${attrName}`] = 1;
              if (attrName === "value") {
                update["$unset"][p + "_type"] = 1;
                update["$unset"][p + "_timestamp"] = 1;
              } else if (attrName === "notification") {
                if (accessList2 == null)
                  update["$unset"][`${p}_attributesTimestamp`] = 1;
              } else if (attrName === "accessList") {
                if (notification2 == null)
                  update["$unset"][`${p}_attributesTimestamp`] = 1;
              }
            }
          }
        }
    }
  }
  update["$unset"] = optimizeProjection(update["$unset"]);
  // Remove overlap possibly caused by parameters changing from objects
  // to regular parameters or vice versa. Reason being that _timestamp
  // represents two different things depending on whether the parameter
  // is an object or not.
  for (const k of Object.keys(update["$unset"]))
    if (update["$set"][k] != null) delete update["$unset"][k];
  // Remove empty keys
  for (const [k, v] of Object.entries(update)) {
    if (k === "$addToSet") {
      for (const [kk, vv] of Object.entries(v))
        if (!vv["$each"].length) delete v[kk];
    } else if (k === "$pull") {
      for (const [kk, vv] of Object.entries(v))
        if (!vv["$in"].length) delete v[kk];
    }
    if (!Object.keys(v).length) delete update[k];
  }
  if (!Object.keys(update).length) return;
  // Mongo doesn't allow $addToSet and $pull at the same time
  let update2;
  if (update["$addToSet"] && update["$pull"]) {
    update2 = { $pull: update["$pull"] };
    delete update["$pull"];
  }
  const result = await collections.devices.updateOne(
    { _id: deviceId },
    update,
    {
      upsert: isNew,
    },
  );
  if (!result.matchedCount && !result.upsertedCount)
    throw new Error(`Device ${deviceId} not found in database`);
  if (update2) {
    await collections.devices.updateOne({ _id: deviceId }, update2);
    return;
  }
 }
 export async function getFaults(
  deviceId: string,
 ): Promise<{ [channel: string]: SessionFault }> {
  const res = await collections.faults
    .find({ _id: { $regex: `^${escapeRegExp(deviceId)}\\:` } })
    .toArray();
  const faults: { [channel: string]: SessionFault } = {};
  for (const r of res) {
    const channel = r._id.slice(deviceId.length + 1);
    const fault: SessionFault = {
      code: r.code,
      message: r.message,
      ...(r.detail && { detail: r.detail }),
      timestamp: +r.timestamp,
      provisions: JSON.parse(r.provisions),
      retries: r.retries,
      ...(r.expiry && { expiry: +r.expiry }),
    };
    faults[channel] = fault;
  }
  return faults;
 }
 export async function saveFault(
  deviceId: string,
  channel: string,
  fault: SessionFault,
 ): Promise<void> {
  const id = `${deviceId}:${channel}`;
  const f: MongoTypes.Fault = {
    _id: id,
    device: deviceId,
    channel: channel,
    timestamp: new Date(fault.timestamp),
    code: fault.code,
    message: fault.message,
    ...(fault.detail && { detail: fault.detail }),
    retries: fault.retries,
    ...(fault.expiry && { expiry: new Date(fault.expiry) }),
    provisions: JSON.stringify(fault.provisions),
  };
  await collections.faults.replaceOne({ _id: id }, f, { upsert: true });
 }
 export async function deleteFault(
  deviceId: string,
  channel: string,
 ): Promise<void> {
  await collections.faults.deleteOne({ _id: `${deviceId}:${channel}` });
 }
 export async function getDueTasks(
  deviceId: string,
  timestamp: number,
 ): Promise<[Task[], number]> {
  const cur = collections.tasks
    .find({ device: deviceId })
    .sort({ timestamp: 1 });
  const tasks = [] as Task[];
  for await (const t of cur) {
    if (+t.timestamp >= timestamp) return [tasks, +t.timestamp];
    const task: Task = {
      _id: t._id.toString(),
      name: t.name,
      ...(t.timestamp && { timestamp: +t.timestamp }),
      ...(t.expiry && { expiry: +t.expiry }),
      ...(t.name === "getParameterValues" && {
        parameterNames: t.parameterNames,
      }),
      ...(t.name === "setParameterValues" && {
        parameterValues: t.parameterValues,
      }),
      ...(t.name === "refreshObject" && {
        objectName: t.objectName,
      }),
      ...(t.name === "download" && {
        fileType: t.fileType,
        fileName: t.fileName,
        targetFileName: t.targetFileName,
      }),
      ...(t.name === "addObject" && {
        objectName: t.objectName,
        parameterValues: t.parameterValues,
      }),
      ...(t.name === "deleteObject" && {
        objectName: t.objectName,
      }),
      ...(t.name === "provisions" && {
        provisions: t.provisions,
      }),
    };
    tasks.push(task);
    // For API compatibility
    if (task.name === "download" && t["file"]) {
      let q;
      if (ObjectId.isValid(t["file"]))
        q = { _id: { $in: [t["file"], new ObjectId(t["file"])] } };
      else q = { _id: t["file"] };
      const res = await collections.files.find(q).toArray();
      if (res[0]) {
        if (!task.fileType) task.fileType = res[0].metadata.fileType;
        if (!task.fileName)
          task.fileName = res[0].filename || res[0]._id.toString();
      }
    }
  }
  return [tasks, null];
 }
 export async function clearTasks(
  deviceId: string,
  taskIds: string[],
 ): Promise<void> {
  await collections.tasks.deleteMany({
    _id: { $in: taskIds.map((id) => new ObjectId(id)) },
  });
 }
 export async function getOperations(
  deviceId: string,
 ): Promise<{ [commandKey: string]: Operation }> {
  const res = await collections.operations
    .find({ _id: { $regex: `^${escapeRegExp(deviceId)}\\:` } })
    .toArray();
  const operations: { [commandKey: string]: Operation } = {};
  for (const r of res) {
    const commandKey = r._id.slice(deviceId.length + 1);
    // Workaround for a bug in v1.2.1 where operation object is saved without deserialization
    if (typeof r.provisions !== "string") {
      delete r._id;
      operations[commandKey] = r as unknown as Operation;
      continue;
    }
    const operation: Operation = {
      name: r.name,
      timestamp: +r.timestamp,
      channels:
        typeof r.channels === "string" ? JSON.parse(r.channels) : r.channels,
      retries: JSON.parse(r.retries),
      provisions: JSON.parse(r.provisions),
      ...(r.args && { args: JSON.parse(r.args) }),
    };
    operations[commandKey] = operation;
  }
  return operations;
 }
 export async function saveOperation(
  deviceId: string,
  commandKey: string,
  operation: Operation,
 ): Promise<void> {
  const id = `${deviceId}:${commandKey}`;
  const o: MongoTypes.Operation = {
    _id: id,
    name: operation.name,
    timestamp: new Date(operation.timestamp),
    channels: JSON.stringify(operation.channels),
    provisions: JSON.stringify(operation.provisions),
    retries: JSON.stringify(operation.retries),
    args: JSON.stringify(operation.args),
  };
  await collections.operations.replaceOne({ _id: id }, o, {
    upsert: true,
  });
 }
 export async function deleteOperation(
  deviceId: string,
  commandKey: string,
 ): Promise<void> {
  await collections.operations.deleteOne({ _id: `${deviceId}:${commandKey}` });
 }
@@ -1,373 +0,0 @@
 import * as vm from "node:vm";
 import * as crypto from "node:crypto";
 import { collections } from "../db/db.ts";
 import { convertOldPrecondition } from "../db/util.ts";
 import * as logger from "../logger.ts";
 import * as scheduling from "../scheduling.ts";
 import Expression, { Value } from "../common/expression.ts";
 import {
  Preset,
  Provisions,
  VirtualParameters,
  Files,
  Config,
 } from "../types.ts";
 import { LocalCache } from "../local-cache.ts";
 interface Snapshot {
  presets: Preset[];
  provisions: Provisions;
  virtualParameters: VirtualParameters;
  files: Files;
  config: Config;
 }
 function flattenObject<T extends Record<keyof T, unknown>>(
  src: T,
  prefix = "",
  dst = {} as T,
 ): T {
  for (const k of Object.keys(src)) {
    const v = src[k];
    if (typeof v === "object" && !Array.isArray(v))
      flattenObject(v as T, `${prefix}${k}.`, dst);
    else dst[`${prefix}${k}`] = v;
  }
  return dst;
 }
 async function fetchPresets(): Promise<[string, Preset[]]> {
  const res = await collections.presets.find().toArray();
  let objects = await collections.objects.find().toArray();
  res.sort((a, b) => (a._id > b._id ? 1 : -1));
  objects.sort((a, b) => (a._id > b._id ? 1 : -1));
  const h = crypto
    .createHash("md5")
    .update(JSON.stringify(res))
    .update(JSON.stringify(objects))
    .digest("hex");
  objects = objects.map((obj) => {
    // Flatten object
    obj = flattenObject(obj);
    // If no keys are defined, consider all parameters as keys to keep the
    // same behavior from v1.0
    if (!obj["_keys"]?.length)
      obj["_keys"] = Object.keys(obj).filter((k) => !k.startsWith("_"));
    return obj;
  });
  res.sort((a, b) => {
    if (a["weight"] === b["weight"])
      return a["_id"] > b["_id"] ? 1 : a["_id"] < b["_id"] ? -1 : 0;
    else return a["weight"] - b["weight"];
  });
  const presets = [] as Preset[];
  for (const preset of res) {
    let schedule: { md5: string; duration: number; schedule: any } = null;
    if (preset["schedule"]) {
      const parts = preset["schedule"].trim().split(/\s+/);
      schedule = {
        md5: crypto.createHash("md5").update(preset["schedule"]).digest("hex"),
        duration: null,
        schedule: null,
      };
      try {
        schedule.duration = +parts.shift() * 1000;
        schedule.schedule = scheduling.parseCron(parts.join(" "));
      } catch {
        logger.warn({
          message: "Invalid preset schedule",
          preset: preset["_id"],
          schedule: preset["schedule"],
        });
        schedule.schedule = false;
      }
    }
    const events = preset["events"] || {};
    let precondition: Expression = new Expression.Literal(true);
    if (preset["precondition"]) {
      try {
        precondition = Expression.parse(preset["precondition"]);
      } catch {
        precondition = convertOldPrecondition(
          JSON.parse(preset["precondition"]),
        );
      }
      // Simplify expression
      precondition = precondition.evaluate((e) => e);
    }
    const _provisions: Preset["provisions"] = [];
    // Generate provisions from the old configuration format
    for (const c of preset.configurations) {
      switch (c.type) {
        case "age":
          _provisions.push([
            "refresh",
            new Expression.Literal(c.name),
            new Expression.Literal(+c.age),
          ]);
          break;
        case "value":
          _provisions.push([
            "value",
            new Expression.Literal(c.name),
            new Expression.Literal(c.value),
          ]);
          break;
        case "add_tag":
          _provisions.push([
            "tag",
            new Expression.Literal(c.tag),
            new Expression.Literal(true),
          ]);
          break;
        case "delete_tag":
          _provisions.push([
            "tag",
            new Expression.Literal(c.tag),
            new Expression.Literal(false),
          ]);
          break;
        case "provision":
          _provisions.push([
            c.name,
            ...(c.args || []).map((a) => Expression.parse(a)),
          ]);
          break;
        case "add_object":
          for (const obj of objects) {
            if (obj["_id"] === c.object) {
              const alias = obj["_keys"]
                .map((k) => `${k}:${JSON.stringify(obj[k])}`)
                .join(",");
              const p = `${c.name}.[${alias}]`;
              _provisions.push([
                "instances",
                new Expression.Literal(p),
                new Expression.Literal(1),
              ]);
              for (const k in obj) {
                if (!k.startsWith("_") && !(obj["_keys"].indexOf(k) !== -1))
                  _provisions.push([
                    "value",
                    new Expression.Literal(`${p}.${k}`),
                    new Expression.Literal(obj[k]),
                  ]);
              }
            }
          }
          break;
        case "delete_object":
          for (const obj of objects) {
            if (obj["_id"] === c.object) {
              const alias = obj["_keys"]
                .map((k) => `${k}:${JSON.stringify(obj[k])}`)
                .join(",");
              const p = `${c.name}.[${alias}]`;
              _provisions.push([
                "instances",
                new Expression.Literal(p),
                new Expression.Literal(0),
              ]);
            }
          }
          break;
        default: {
          const exhaustiveCheck: never = c;
          throw new Error(
            `Unknown configuration type ${exhaustiveCheck["type"]}`,
          );
        }
      }
    }
    presets.push({
      name: preset["_id"],
      channel: (preset["channel"] as string) || "default",
      schedule: schedule,
      events: events,
      precondition: precondition,
      provisions: _provisions,
    });
  }
  return [h, presets];
 }
 async function fetchProvisions(): Promise<[string, Provisions]> {
  const res = await collections.provisions.find().toArray();
  res.sort((a, b) => (a._id > b._id ? 1 : -1));
  const h = crypto.createHash("md5").update(JSON.stringify(res)).digest("hex");
  const provisions = {};
  for (const r of res) {
    provisions[r._id] = {};
    provisions[r._id].md5 = crypto
      .createHash("md5")
      .update(r.script)
      .digest("hex");
    provisions[r._id].script = new vm.Script(
      `"use strict";(function(){\n${r.script}\n})();`,
      { filename: r._id, lineOffset: -1 },
    );
  }
  return [h, provisions];
 }
 async function fetchVirtualParameters(): Promise<[string, VirtualParameters]> {
  const res = await collections.virtualParameters.find().toArray();
  res.sort((a, b) => (a._id > b._id ? 1 : -1));
  const h = crypto.createHash("md5").update(JSON.stringify(res)).digest("hex");
  const virtualParameters = {};
  for (const r of res) {
    virtualParameters[r._id] = {};
    virtualParameters[r._id].md5 = crypto
      .createHash("md5")
      .update(r.script)
      .digest("hex");
    virtualParameters[r._id].script = new vm.Script(
      `"use strict";(function(){\n${r.script}\n})();`,
      { filename: r._id, lineOffset: -1 },
    );
  }
  return [h, virtualParameters];
 }
 async function fetchFiles(): Promise<[string, Files]> {
  const res = await collections.files.find().toArray();
  res.sort((a, b) => (a._id > b._id ? 1 : -1));
  const h = crypto.createHash("md5").update(JSON.stringify(res)).digest("hex");
  const files = {};
  for (const r of res) {
    const id = r.filename || r._id.toString();
    files[id] = {};
    files[id].length = r.length;
  }
  return [h, files];
 }
 async function fetchConfig(): Promise<[string, Config]> {
  const conf = await collections.config.find().toArray();
  conf.sort((a, b) => (a._id > b._id ? 1 : -1));
  const h = crypto.createHash("md5").update(JSON.stringify(conf)).digest("hex");
  const _config = {};
  for (const c of conf) {
    // Evaluate expressions to simplify them
    _config[c._id] = Expression.parse(c.value).evaluate((e) => e);
  }
  return [h, _config];
 }
 const localCache = new LocalCache<Snapshot>("cwmp-local-cache-hash", refresh);
 async function refresh(): Promise<[string, Snapshot]> {
  const res = await Promise.all([
    fetchPresets(),
    fetchProvisions(),
    fetchVirtualParameters(),
    fetchFiles(),
    fetchConfig(),
  ]);
  const h = crypto.createHash("md5");
  for (const r of res) h.update(r[0]);
  const snapshot = {
    presets: res[0][1],
    provisions: res[1][1],
    virtualParameters: res[2][1],
    files: res[3][1],
    config: res[4][1],
  };
  return [h.digest("hex"), snapshot];
 }
 export async function getRevision(): Promise<string> {
  return await localCache.getRevision();
 }
 export function getPresets(revision: string): Preset[] {
  return localCache.get(revision).presets;
 }
 export function getProvisions(revision: string): Provisions {
  return localCache.get(revision).provisions;
 }
 export function getVirtualParameters(revision: string): VirtualParameters {
  return localCache.get(revision).virtualParameters;
 }
 export function getFiles(revision: string): Files {
  return localCache.get(revision).files;
 }
 export function getConfig(
  revision: string,
  key: string,
  dflt: string,
  fn: (e: Expression) => Expression.Literal,
 ): string;
 export function getConfig(
  revision: string,
  key: string,
  dflt: number,
  fn: (e: Expression) => Expression.Literal,
 ): number;
 export function getConfig(
  revision: string,
  key: string,
  dflt: boolean,
  fn: (e: Expression) => Expression.Literal,
 ): boolean;
 export function getConfig(
  revision: string,
  key: string,
  dflt: Value,
  fn: (e: Expression) => Expression.Literal,
 ): Value {
  const snapshot = localCache.get(revision);
  if (!snapshot) throw new Error("Cache snapshot does not exist");
  const e = snapshot.config[key];
  if (!e) return dflt;
  const v = e.evaluate(fn).value;
  if (typeof v !== typeof dflt) return dflt;
  return v;
 }
 export function getConfigExpression(
  snapshotKey: string,
  key: string,
 ): Expression {
  const snapshot = localCache.get(snapshotKey);
  return snapshot.config[key];
 }
@@ -0,0 +1,230 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 ###
 config = require './config'
 mongodb = require 'mongodb'
 redis = require('redis')
 parameters = require './parameters'
 redisClient = null
 tasksCollection = null
 devicesCollection = null
 presetsCollection = null
 objectsCollection = null
 # Only flush redis DB if aliases or custom commands config has changed
 flushRedis = (callback) ->
  redisClient.get('config_hash', (err, res) ->
    return callback(err) if err
    hash = require('crypto')
      .createHash('md5')
      .update(JSON.stringify(config.PARAMETERS))
      .update(JSON.stringify(config.CUSTOM_COMMANDS))
      .digest('hex')
    if res == hash
      return callback()
    redisClient.flushdb((err) ->
      return callback(err) if err
      redisClient.set('config_hash', hash, (err, res) ->
        return callback(err)
      )
    )
  )
 connect = (callback) ->
  callbackCounter = 6
  mongodb.MongoClient.connect(config.get('MONGODB_CONNECTION_URL'), {db:{w:1},server:{autoReconnect:true}}, (err, db) ->
    return callback(err) if err
    exports.mongoDb = db
    db.collection('tasks', (err, collection) ->
      exports.tasksCollection = tasksCollection = collection
      collection?.ensureIndex({device: 1, timestamp: 1}, (err) ->
      )
      if --callbackCounter == 0 or err
        callbackCounter = 0
        return callback(err)
    )
    db.collection('devices', (err, collection) ->
      exports.devicesCollection  = devicesCollection = collection
      if --callbackCounter == 0 or err
        callbackCounter = 0
        return callback(err)
    )
    db.collection('presets', (err, collection) ->
      exports.presetsCollection = presetsCollection = collection
      if --callbackCounter == 0 or err
        callbackCounter = 0
        return callback(err)
    )
    db.collection('objects', (err, collection) ->
      exports.objectsCollection = objectsCollection = collection
      if --callbackCounter == 0 or err
        callbackCounter = 0
        return callback(err)
    )
    db.collection('fs.files', (err, collection) ->
      exports.filesCollection = filesCollection = collection
      if --callbackCounter == 0 or err
        callbackCounter = 0
        return callback(err)
    )
    exports.redisClient = redisClient = redis.createClient(config.get('REDIS_PORT'), config.get('REDIS_HOST'))
    redisClient.select(config.get('REDIS_DB'), (err) ->
      if err
        callbackCounter = 0
        return callback(err)
      flushRedis((err) ->
        if --callbackCounter == 0 or err
          callbackCounter = 0
          return callback(err)
      )
    )
  )
 disconnect = () ->
  exports.mongoDb.close()
  redisClient.quit()
 getTask = (taskId, callback) ->
  tid = String(taskId)
  redisClient.get(tid, (err, res) ->
    return callback(err) if err
    if res?
      callback(err, JSON.parse(res))
    else
      tasksCollection.findOne({_id : mongodb.ObjectID(tid)}, (err, task) ->
        callback(err, task)
      )
  )
 getCached = (name, valueCallback, valueExpiry, callback) ->
  redisClient.get(name, (err, res) ->
    return callback(err) if err
    return callback(null, JSON.parse(res)) if res?
    lockAcquired = () ->
      valueCallback((err, value) ->
        return callback(err, value) if err
        redisClient.setex(name, valueExpiry, JSON.stringify(value), (err) ->
          return callback(err, value) if err
          redisClient.del("lock.#{name}", (err) ->
            callback(err, value)
          )
        )
      )
    redisClient.setnx("lock.#{name}", Date.now() + 30000, (err, res) ->
      return callback(err) if err
      if res == 1
        lockAcquired()
      else
        setTimeout(() ->
          redisClient.get("lock.#{name}", (err, timestamp) ->
            return callback(err) if err
            now = Date.now()
            if not timestamp?
              getCached(name, valueCallback, valueExpiry, callback)
            else if timestamp > now
              setTimeout(() ->
                getCached(name, valueCallback, valueExpiry, callback)
              , 1000)
            else
              redisClient.getset("lock.#{name}", now + 30000, (err, timestamp) ->
                return callback(err) if err
                if timestamp > now
                  setTimeout(() ->
                    getCached(name, valueCallback, valueExpiry, callback)
                  , 1000)
                else
                  lockAcquired()
              )
          )
        , 1000)
    )
  )
 getAliases = (callback) ->
  getCached('aliases', parameters.compileAliases, config.get('PRESETS_CACHE_DURATION'), (err, res) ->
    throw err if err
    callback(res)
  )
 getPresetsObjectsAliases = (callback) ->
  redisClient.mget('presets', 'objects', 'aliases', (err, res) ->
    presets = JSON.parse(res[0])
    objects = JSON.parse(res[1])
    aliases = JSON.parse(res[2])
    if presets and objects and aliases
      return callback(presets, objects, aliases)
    if not presets
      getCached('presets', (callback) ->
        presetsCollection.find().toArray((err, res) ->
          callback(err, res)
        )
      , config.get('PRESETS_CACHE_DURATION'), (err, res) ->
        throw err if err
        presets = res
        callback(presets, objects, aliases) if objects and aliases
      )
    if not objects
      getCached('objects', (callback) ->
        objectsCollection.find().toArray((err, res) ->
          return callback(err) if err
          objs = {}
          objs[r._id] = r for r in res
          callback(null, objs)
        )
      , config.get('PRESETS_CACHE_DURATION'), (err, res) ->
        throw err if err
        objects = res
        callback(presets, objects, aliases) if presets and aliases
      )
    if not aliases
      getCached('aliases', parameters.compileAliases, config.get('PRESETS_CACHE_DURATION'), (err, res) ->
        throw err if err
        aliases = res
        callback(presets, objects, aliases) if presets and objects
      )
  )
 exports.getTask = getTask
 exports.getPresetsObjectsAliases = getPresetsObjectsAliases
 exports.getAliases = getAliases
 exports.connect = connect
 exports.disconnect = disconnect
@@ -1,57 +0,0 @@
 import { MongoClient, Collection, GridFSBucket } from "mongodb";
 import { get } from "../config.ts";
 import * as MongoTypes from "./types.ts";
 export let filesBucket: GridFSBucket;
 export const collections = {
  devices: null as Collection<MongoTypes.Device>,
  presets: null as Collection<MongoTypes.Preset>,
  objects: null as Collection<MongoTypes.Object>,
  provisions: null as Collection<MongoTypes.Provision>,
  virtualParameters: null as Collection<MongoTypes.VirtualParameter>,
  faults: null as Collection<MongoTypes.Fault>,
  tasks: null as Collection<MongoTypes.Task>,
  files: null as Collection<MongoTypes.File>,
  operations: null as Collection<MongoTypes.Operation>,
  permissions: null as Collection<MongoTypes.Permission>,
  users: null as Collection<MongoTypes.User>,
  config: null as Collection<MongoTypes.Config>,
  cache: null as Collection<MongoTypes.Cache>,
  locks: null as Collection<MongoTypes.Lock>,
 };
 let clientPromise: Promise<MongoClient>;
 export async function connect(): Promise<void> {
  clientPromise = MongoClient.connect("" + get("MONGODB_CONNECTION_URL"));
  const client = await clientPromise;
  const db = client.db();
  collections.tasks = db.collection("tasks");
  collections.devices = db.collection("devices");
  collections.presets = db.collection("presets");
  collections.objects = db.collection("objects");
  collections.files = db.collection("fs.files");
  collections.provisions = db.collection("provisions");
  collections.virtualParameters = db.collection("virtualParameters");
  collections.faults = db.collection("faults");
  collections.operations = db.collection("operations");
  collections.permissions = db.collection("permissions");
  collections.users = db.collection("users");
  collections.config = db.collection("config");
  collections.cache = db.collection("cache");
  collections.locks = db.collection("locks");
  filesBucket = new GridFSBucket(db);
  await Promise.all([
    collections.tasks.createIndex({ device: 1, timestamp: 1 }),
    collections.cache.createIndex({ expire: 1 }, { expireAfterSeconds: 0 }),
    collections.locks.createIndex({ expire: 1 }, { expireAfterSeconds: 0 }),
  ]);
 }
 export async function disconnect(): Promise<void> {
  if (clientPromise != null) await (await clientPromise).close();
 }
@@ -1,660 +0,0 @@
 import { Filter } from "mongodb";
 import { EJSON } from "bson";
 import { complement } from "espresso-iisojs";
 import { parseLikePattern } from "../common/expression/parser.ts";
 import Expression from "../common/expression.ts";
 import { decodeTag } from "../util.ts";
 import {
  SynthContextBase,
  likeDisjoint,
  likeImplies,
  Clause,
 } from "../common/expression/synth.ts";
 import normalize from "../common/expression/normalize.ts";
 type Minterm = number[];
 function getParam(exp: Expression, collection: string): string {
  if (!(exp instanceof Expression.Parameter))
    throw new Error("Left-hand operand must be a parameter");
  const p = exp.path.toString();
  if (collection === "devices") {
    if (p === "DeviceID.ID") return "_id";
    else if (p === "DeviceID") return "_deviceId";
    else if (p.startsWith("DeviceID.")) return "_deviceId._" + p.slice(9);
    else if (p === "Events.Inform") return "_lastInform";
    else if (p === "Events.Registered") return "_registered";
    else if (p === "Events.0_BOOTSTRAP") return "_lastBootstrap";
    else if (p === "Events.1_BOOT") return "_lastBoot";
    else if (!p.endsWith("._value") && !p.startsWith("Tags."))
      return `${p}._value`;
  }
  return p;
 }
 function getTypes(parameter: string, collection: string): string[] {
  if (collection === "devices") {
    if (parameter === "_id") return ["string"];
    if (parameter === "_lastInform") return ["date"];
    if (parameter === "_registered") return ["date"];
    if (parameter === "_lastBootstrap") return ["date"];
    if (parameter === "_lastBoot") return ["date"];
    if (parameter.startsWith("_deviceId.")) return ["string"];
    if (parameter === "Reboot._value") return ["date"];
    if (parameter === "FactoryReset._value") return ["date"];
    if (parameter.endsWith("_timestamp")) return ["date"];
    if (parameter.startsWith("Downloads.")) {
      if (parameter.endsWith("Download._value")) return ["date"];
      if (parameter.endsWith("Time._value")) return ["date"];
      if (parameter.endsWith("Name._value")) return ["string"];
      if (parameter.endsWith("Type._value")) return ["string"];
    }
    if (parameter.endsWith("_value"))
      return ["bool", "number", "date", "string"];
  } else if (collection === "tasks") {
    if (parameter === "_id") return ["oid"];
    if (parameter === "timestamp") return ["date"];
    if (parameter === "expiry") return ["date"];
    if (parameter === "name") return ["string"];
    if (parameter === "device") return ["string"];
  } else if (collection === "faults") {
    if (parameter === "_id") return ["string"];
    if (parameter === "timestamp") return ["date"];
    if (parameter === "expiry") return ["date"];
    if (parameter === "code") return ["string"];
    if (parameter === "retries") return ["number"];
    if (parameter === "channel") return ["string"];
    if (parameter === "device") return ["string"];
    if (parameter === "message") return ["string"];
  } else if (collection === "users") {
    if (parameter === "_id") return ["string"];
    if (parameter === "roles") return ["string"];
    else if (parameter === "password")
      throw new Error("Cannot query restricted parameters");
    else if (parameter === "salt")
      throw new Error("Cannot query restricted parameters");
  } else if (collection === "config") {
    if (parameter === "_id") return ["string"];
    else if (parameter === "value") return ["string"];
  } else if (collection === "files") {
    if (parameter === "_id") return ["string"];
    if (parameter === "metadata.fileType") return ["string"];
    if (parameter === "metadata.oui") return ["string"];
    if (parameter === "metadata.productClass") return ["string"];
    if (parameter === "metadata.version") return ["string"];
  } else if (collection === "permissions") {
    if (parameter === "_id") return ["string"];
    if (parameter === "role") return ["string"];
    if (parameter === "resource") return ["devices"];
    if (parameter === "access") return ["number"];
    if (parameter === "filter") return ["string"];
    if (parameter === "validate") return ["string"];
  } else if (collection === "presets") {
    if (parameter === "_id") return ["string"];
    if (parameter === "weight") return ["number"];
    if (parameter === "channel") return ["string"];
    if (parameter === "precondition") return ["string"];
    if (parameter.startsWith("events.")) return ["bool"];
  } else if (collection === "provisions") {
    if (parameter === "_id") return ["string"];
    if (parameter === "script") return ["string"];
  } else if (collection === "virtualParameters") {
    if (parameter === "_id") return ["string"];
    if (parameter === "script") return ["string"];
  }
  if (parameter === "_id") return ["oid", "string"];
  return ["bool", "number", "date", "string"];
 }
 function roundOid(oid: string, roundUp: boolean): string {
  const match = (oid.match(/^[0-9a-f]*/)?.[0] ?? "").slice(0, 24);
  let num = BigInt("0x0" + match);
  let lastChar = 0;
  if (oid.length > match.length) lastChar += oid.charCodeAt(match.length);
  if (match.length < 24) lastChar -= 48;
  if (lastChar > 0 && roundUp) num++;
  num <<= BigInt(4 * (24 - match.length));
  if (lastChar < 0 && !roundUp) --num;
  const str = num.toString(16);
  if (str.length > 24 || str.startsWith("-")) return null;
  return str.padStart(24, "0");
 }
 function groupBy<T, K>(
  input: T[],
  callback: (item: T) => K,
 ): Iterable<[K, T[]]> {
  const groups = new Map<K, T[]>();
  for (const item of input) {
    const key = callback(item);
    let arr = groups.get(key);
    if (!arr) groups.set(key, (arr = []));
    arr.push(item);
  }
  return groups.entries();
 }
 abstract class MongoClause {
  abstract readonly parameter: string;
  abstract toQuery(truthy: boolean): Filter<unknown>;
  toString(): string {
    return JSON.stringify(this.toQuery(true));
  }
 }
 class MongoClauseArray extends MongoClause {
  constructor(
    public readonly parameter: string,
    public readonly value: string,
  ) {
    super();
  }
  toQuery(truthy: boolean): Filter<unknown> {
    if (truthy) return { [this.parameter]: { $eq: this.value } };
    else return { [this.parameter]: { $ne: this.value } };
  }
 }
 class MongoClauseCompare<T> extends MongoClause {
  constructor(
    public readonly parameter: string,
    public readonly op: "$eq" | "$gt" | "$lt" | "$gte" | "$lte",
    public readonly value: T,
    public readonly type: "" | "bool" | "number" | "string" | "date" | "oid",
  ) {
    super();
  }
  toQuery(truthy: boolean): Filter<unknown> {
    let v: any = this.value;
    if (this.type === "date" && typeof v === "number")
      v = { $date: new Date(v).toISOString() };
    if (this.type === "oid" && typeof v === "string") v = { $oid: v };
    if (truthy) return { [this.parameter]: { [this.op]: v } };
    else if (this.op === "$eq") return { [this.parameter]: { $ne: v } };
    else return { [this.parameter]: { $not: { [this.op]: v } } };
  }
 }
 class MongoClauseType extends MongoClause {
  constructor(
    public readonly parameter: string,
    public readonly type: string,
  ) {
    super();
  }
  toQuery(truthy: boolean): Filter<unknown> {
    if (truthy) return { [this.parameter]: { $type: this.type } };
    else return { [this.parameter]: { $not: { $type: this.type } } };
  }
 }
 class MongoClauseLike extends MongoClause {
  readonly pattern: string[];
  constructor(
    public readonly parameter: string,
    pat: string,
    esc: string,
    public readonly caseSensitive: boolean,
  ) {
    super();
    this.pattern = parseLikePattern(pat, esc);
  }
  toQuery(truthy: boolean): Filter<unknown> {
    const convChars = {
      "-": "\\-",
      "/": "\\/",
      "\\": "\\/",
      "^": "\\^",
      $: "\\$",
      "*": "\\*",
      "+": "\\+",
      "?": "\\?",
      ".": "\\.",
      "(": "\\(",
      ")": "\\)",
      "|": "\\|",
      "[": "\\[",
      "]": "\\]",
      "{": "\\{",
      "}": "\\}",
      "\\%": ".*",
      "\\_": ".",
    };
    const chars = this.pattern.map((c) => convChars[c] ?? c);
    chars[0] = chars[0] === ".*" ? "" : "^" + chars[0];
    const l = chars.length - 1;
    chars[l] = [".*", ""].includes(chars[l]) ? "" : chars[l] + "$";
    const pattern = chars.join("");
    const options = this.caseSensitive ? "s" : "is";
    if (truthy) {
      return { [this.parameter]: { $regularExpression: { options, pattern } } };
    } else {
      return {
        [this.parameter]: {
          $not: { $regularExpression: { options, pattern } },
        },
      };
    }
  }
 }
 class MongoSynthContext extends SynthContextBase<Clause, MongoClause> {
  constructor(private readonly collection: string) {
    super();
  }
  getMinterms(clause: Clause, res: number): number[][] {
    res = res & 0b111;
    if (res & (res - 1)) return complement(this.getMinterms(clause, ~res));
    const exp = clause.expression();
    if (res === 0b001) {
      const minterms: number[][] = [];
      for (const dep of clause.getNullables()) {
        const e = dep.operand.expression();
        const param = getParam(e, this.collection);
        if (param.startsWith("Tags.") && this.collection === "devices") {
          const t = decodeTag(param.slice(5));
          const c = new MongoClauseArray("_tags", t);
          minterms.push([this.getVar(c) << 1]);
          continue;
        }
        const c = new MongoClauseCompare(param, "$eq", null, "");
        minterms.push([(this.getVar(c) << 1) ^ 1]);
      }
      return minterms;
    }
    if (!(exp instanceof Expression.Binary))
      throw new Error("Invalid query expression");
    const truthy = res === 0b100;
    if ([">", "<", "="].includes(exp.operator)) {
      const param = getParam(exp.left, this.collection);
      if (!(exp.right instanceof Expression.Literal))
        throw new Error(`Right-hand operand must be a literal value`);
      let rhs = exp.right.value;
      if (typeof rhs === "boolean") rhs = +rhs;
      if (
        param.startsWith("Tags.") &&
        this.collection === "devices" &&
        exp.operator === "="
      ) {
        const t = decodeTag(param.slice(5));
        const c = new MongoClauseArray("_tags", t);
        if (typeof rhs === "string") rhs = 2;
        if (exp.operator === "=") {
          if ((rhs === 1) !== truthy) return [];
        } else if (exp.operator === ">") {
          if ((truthy && rhs >= 1) || (!truthy && rhs < 1)) return [];
        } else if (exp.operator === "<") {
          if ((truthy && rhs <= 1) || (!truthy && rhs > 1)) return [];
        }
        return [[(this.getVar(c) << 1) ^ 1]];
      }
      let op: "$gt" | "$lt" | "$eq" | "$gte" | "$lte";
      if (exp.operator === "=") op = "$eq";
      else if (exp.operator === ">") op = truthy ? "$gt" : "$lte";
      else if (exp.operator === "<") op = truthy ? "$lt" : "$gte";
      const possibleTypes = new Set(getTypes(param, this.collection));
      const clauses: MongoClause[] = [];
      if (typeof rhs === "number") {
        if (possibleTypes.has("number"))
          clauses.push(new MongoClauseCompare(param, op, rhs, "number"));
        if (possibleTypes.has("date"))
          clauses.push(new MongoClauseCompare(param, op, rhs, "date"));
        if (possibleTypes.has("bool") && (rhs === 0 || rhs === 1))
          clauses.push(new MongoClauseCompare(param, op, !!rhs, "bool"));
      } else if (typeof rhs === "string") {
        if (possibleTypes.has("string"))
          clauses.push(new MongoClauseCompare(param, op, rhs, "string"));
        if (possibleTypes.has("oid")) {
          const oid = roundOid(rhs, op === "$lt" || op === "$lte");
          if (oid && (op !== "$eq" || oid === rhs))
            clauses.push(new MongoClauseCompare(param, op, oid, "oid"));
        }
      }
      if (op === "$eq" && !truthy) {
        clauses.push(new MongoClauseCompare(param, "$eq", null, ""));
        return [clauses.map((c) => this.getVar(c) << 1)];
      }
      // In the following clauses we could use $type operator, but we want the
      // final query to use comparison operators to check for type because the
      // $type operator in MongoDB doesn't use indexes
      if (typeof rhs === "number") {
        if (possibleTypes.has("bool")) {
          if (
            (rhs > 1 && (op === "$lt" || op === "$lte")) ||
            (rhs < 0 && (op === "$gt" || op === "$gte"))
          )
            clauses.push(new MongoClauseCompare(param, "$gte", false, "bool"));
        }
        if (op === "$gt" || op === "$gte") {
          if (possibleTypes.has("string"))
            clauses.push(new MongoClauseCompare(param, "$gte", "", "string"));
          if (possibleTypes.has("oid")) {
            clauses.push(
              new MongoClauseCompare(
                param,
                "$gte",
                "000000000000000000000000",
                "oid",
              ),
            );
          }
        }
      } else if (typeof rhs === "string") {
        if (op === "$lt" || op === "$lte") {
          if (possibleTypes.has("bool"))
            clauses.push(new MongoClauseCompare(param, "$gte", false, "bool"));
          if (possibleTypes.has("number")) {
            clauses.push(new MongoClauseCompare(param, "$gte", 0, "number"));
            clauses.push(new MongoClauseCompare(param, "$lt", 0, "number"));
          }
          if (possibleTypes.has("date")) {
            clauses.push(new MongoClauseCompare(param, "$gte", 0, "date"));
            clauses.push(new MongoClauseCompare(param, "$lt", 0, "date"));
          }
        }
      }
      return clauses.map((c) => [(this.getVar(c) << 1) ^ 1]);
    } else if (exp.operator === "LIKE") {
      if (
        !(
          exp.right instanceof Expression.Literal &&
          typeof exp.right.value === "string"
        )
      )
        throw new Error("Right-hand operand of 'LIKE' must be a string");
      const pat = exp.right.value;
      let p = exp.left;
      let caseSensitive = true;
      if (
        p instanceof Expression.FunctionCall &&
        ["UPPER", "LOWER"].includes(p.name)
      ) {
        if (p.name === "UPPER" && pat !== pat.toUpperCase())
          return truthy ? [] : [[]];
        if (p.name === "LOWER" && pat !== pat.toLowerCase())
          return truthy ? [] : [[]];
        caseSensitive = false;
        p = p.args[0];
      }
      const param = getParam(p, this.collection);
      const c = new MongoClauseLike(param, pat, null, caseSensitive);
      if (truthy) return [[(this.getVar(c) << 1) ^ 1]];
      const typeClause = new MongoClauseCompare(param, "$gte", "", "string");
      const r = [[this.getVar(c) << 1, (this.getVar(typeClause) << 1) ^ 1]];
      return r;
    } else {
      throw new Error("Invalid query expression");
    }
  }
  getDcSet(minterms: Minterm[]): number[][] {
    const dcSet: number[][] = [];
    const vars = new Set(minterms.flat().map((l) => l >> 1));
    const clauses = Array.from(vars).map((v) => this.getClause(v));
    for (const [parameter, clauses2] of groupBy(clauses, (c) => c.parameter)) {
      const comparisons = clauses2.filter(
        (c) => c instanceof MongoClauseCompare && c.value !== null,
      ) as MongoClauseCompare<unknown>[];
      for (const [type, clauses3] of groupBy(comparisons, (c) => c.type)) {
        const isType = this.getVar(new MongoClauseType(parameter, type));
        const values = new Set(clauses3.map((c) => c.value));
        const valuesSorted = Array.from(values).sort((a, b) =>
          a > b ? 1 : -1,
        );
        for (const [i, v] of valuesSorted.entries()) {
          const eq = this.getVar(
            new MongoClauseCompare(parameter, "$eq", v, type),
          );
          const gt = this.getVar(
            new MongoClauseCompare(parameter, "$gt", v, type),
          );
          const lt = this.getVar(
            new MongoClauseCompare(parameter, "$lt", v, type),
          );
          const gte = this.getVar(
            new MongoClauseCompare(parameter, "$gte", v, type),
          );
          const lte = this.getVar(
            new MongoClauseCompare(parameter, "$lte", v, type),
          );
          if (type === "bool") {
            if (v === false) dcSet.push([(lt << 1) ^ 1]);
            else if (v === true) dcSet.push([(gt << 1) ^ 1]);
          } else if (type === "string") {
            if (v === "") dcSet.push([(lt << 1) ^ 1]);
          } else if (type === "oid") {
            if (v < "000000000000000000000000") dcSet.push([(lte << 1) ^ 1]);
            else if (v === "000000000000000000000000")
              dcSet.push([(lt << 1) ^ 1]);
            else if (v > "ffffffffffffffffffffffff")
              dcSet.push([(gte << 1) ^ 1]);
            else if (v === "ffffffffffffffffffffffff")
              dcSet.push([(gt << 1) ^ 1]);
          }
          dcSet.push([(lt << 1) ^ 1, (gte << 1) ^ 1]);
          dcSet.push([(gt << 1) ^ 1, (gte << 1) ^ 0]);
          dcSet.push([(eq << 1) ^ 0, (lt << 1) ^ 0, (lte << 1) ^ 1]);
          dcSet.push([(eq << 1) ^ 1, (gte << 1) ^ 0]);
          dcSet.push([(eq << 1) ^ 1, (gt << 1) ^ 1]);
          if (i === 0) {
            dcSet.push([(isType << 1) ^ 0, (gte << 1) ^ 1]);
            dcSet.push([(isType << 1) ^ 0, (lt << 1) ^ 1]);
          } else {
            const gt2 = this.getVar(
              new MongoClauseCompare(
                parameter,
                "$gt",
                valuesSorted[i - 1],
                type,
              ),
            );
            const lte2 = this.getVar(
              new MongoClauseCompare(
                parameter,
                "$lte",
                valuesSorted[i - 1],
                type,
              ),
            );
            dcSet.push([(gt2 << 1) ^ 0, (gte << 1) ^ 1]);
            dcSet.push([(gt2 << 1) ^ 0, (lte2 << 1) ^ 0, (lt << 1) ^ 1]);
          }
          if (i === valuesSorted.length - 1)
            dcSet.push([(isType << 1) ^ 1, (gt << 1) ^ 0, (lte << 1) ^ 0]);
        }
      }
      const likes = clauses2.filter(
        (c) => c instanceof MongoClauseLike,
      ) as MongoClauseLike[];
      if (likes.length) {
        const isType = this.getVar(new MongoClauseType(parameter, "string"));
        for (let i1 = 0; i1 < likes.length; ++i1) {
          const l1 = likes[i1];
          let p1 = l1.pattern;
          dcSet.push([(this.getVar(l1) << 1) ^ 1, (isType << 1) ^ 0]);
          for (let i2 = i1 + 1; i2 < likes.length; ++i2) {
            const l2 = likes[i2];
            let p2 = l2.pattern;
            if (!l1.caseSensitive || !l2.caseSensitive) {
              p1 = p1.map((c) => c.toLowerCase());
              p2 = p2.map((c) => c.toLowerCase());
            }
            if (likeDisjoint(p1, p2)) {
              dcSet.push([
                (this.getVar(l1) << 1) ^ 1,
                (this.getVar(l2) << 1) ^ 1,
              ]);
            } else if (
              (!l1.caseSensitive || l2.caseSensitive) &&
              likeImplies(p1, p2)
            ) {
              dcSet.push([
                (this.getVar(l1) << 1) ^ 0,
                (this.getVar(l2) << 1) ^ 1,
              ]);
            } else if (
              (!l2.caseSensitive || l1.caseSensitive) &&
              likeImplies(p2, p1)
            ) {
              dcSet.push([
                (this.getVar(l1) << 1) ^ 1,
                (this.getVar(l2) << 1) ^ 0,
              ]);
            }
          }
        }
      }
      const isNull = this.getVar(
        new MongoClauseCompare(parameter, "$eq", null, ""),
      );
      const types = getTypes(parameter, this.collection).map((t) =>
        this.getVar(new MongoClauseType(parameter, t)),
      );
      dcSet.push([(isNull << 1) ^ 0, ...types.map((t) => (t << 1) ^ 0)]);
      for (let i = 0; i < types.length; ++i) {
        const t1 = types[i];
        dcSet.push([(t1 << 1) ^ 1, (isNull << 1) ^ 1]);
        for (let j = i + 1; j < types.length; ++j) {
          const t2 = types[j];
          dcSet.push([(t1 << 1) ^ 1, (t2 << 1) ^ 1]);
        }
      }
      if (parameter === "_id") dcSet.push([(isNull << 1) ^ 1]);
    }
    return dcSet;
  }
  canRaise(i: number, s: Set<number>): boolean {
    if (!(i & 1)) return true;
    const c = this.getClause(i >> 1);
    if (c instanceof MongoClauseCompare) {
      for (const j of s) {
        if (j === i) continue;
        const c2 = this.getClause(j >> 1);
        if (c2.parameter !== c.parameter) continue;
        if (!(j & 1)) return false;
        if (c2 instanceof MongoClauseType) return false;
      }
    }
    return true;
  }
  toQuery(minterms: Minterm[]): Filter<unknown> {
    const or = [];
    const ejsonOps = ["$oid", "$date", "$regularExpression"];
    for (const minterm of minterms) {
      const query = {};
      loop: for (const clause of minterm) {
        if (!minterm.length) return {};
        const negate = !!(clause & 1);
        const c = this.getClause(clause >> 1);
        const q = c.toQuery(negate);
        if (Object.keys(q).length !== 1)
          throw new Error("Invalid query expression");
        const [param, value] = Object.entries(q)[0];
        const dests = [query, ...(query["$and"] ?? [])];
        for (const dest of dests) {
          if (!(param in dest)) {
            dest[param] = value;
            continue loop;
          }
          let src = value;
          let dst = dest[param];
          if (Object.getPrototypeOf(src).constructor !== Object) continue;
          if (Object.getPrototypeOf(dst).constructor !== Object) continue;
          let srcKeys = Object.keys(src);
          let dstKeys = Object.keys(dst);
          if ([...srcKeys, ...dstKeys].every((k) => k === "$not")) {
            src = src["$not"];
            dst = dst["$not"];
            if (Object.getPrototypeOf(src).constructor !== Object) continue;
            if (Object.getPrototypeOf(dst).constructor !== Object) continue;
            srcKeys = Object.keys(src);
            dstKeys = Object.keys(dst);
          }
          if (srcKeys.some((k) => dstKeys.includes(k))) continue;
          // Don't mix regular operators with EJSON special operators
          if (srcKeys.some((k) => ejsonOps.includes(k))) continue;
          if (dstKeys.some((k) => ejsonOps.includes(k))) continue;
          Object.assign(dst, src);
          continue loop;
        }
        query["$and"] ??= [];
        query["$and"].push({ [param]: value });
      }
      or.push(query);
    }
    if (or.length === 1) return or[0];
    return { $or: or };
  }
 }
 export function toMongoQuery(
  exp: Expression,
  resource: string,
 ): Filter<unknown> | false {
  exp = normalize(exp);
  const clause = Clause.fromExpression(normalize(exp));
  const context = new MongoSynthContext(resource);
  const minterms = clause.getMinterms(context, 0b100);
  const minimized = context.minimize(minterms);
  if (!minimized.length) return false;
  return EJSON.deserialize(context.toQuery(minimized));
 }
 export function validQuery(exp: Expression, resource: string): void {
  const clause = Clause.fromExpression(normalize(exp));
  const context = new MongoSynthContext(resource);
  clause.getMinterms(context, 0b100);
 }
@@ -1,184 +0,0 @@
 import { ObjectId } from "mongodb";
 import { FaultStruct } from "../types.ts";
 import { Value } from "../common/expression.ts";
 export interface Fault {
  _id: string;
  device: string;
  channel: string;
  timestamp: Date;
  provisions: string;
  retries: number;
  code: string;
  message: string;
  detail?:
    | FaultStruct
    | {
        name: string;
        message: string;
        stack?: string;
      };
  expiry?: Date;
 }
 interface TaskBase {
  _id: ObjectId;
  timestamp?: Date;
  expiry?: Date;
  name: string;
  device: string;
 }
 interface TaskGetParameterValues extends TaskBase {
  name: "getParameterValues";
  parameterNames: string[];
 }
 interface TaskSetParameterValues extends TaskBase {
  name: "setParameterValues";
  parameterValues: [string, string | number | boolean, string?][];
 }
 interface TaskRefreshObject extends TaskBase {
  name: "refreshObject";
  objectName: string;
 }
 interface TaskReboot extends TaskBase {
  name: "reboot";
 }
 interface TaskFactoryReset extends TaskBase {
  name: "factoryReset";
 }
 interface TaskDownload extends TaskBase {
  name: "download";
  fileType: string;
  fileName: string;
  targetFileName?: string;
 }
 interface TaskAddObject extends TaskBase {
  name: "addObject";
  objectName: string;
  parameterValues: [string, string | number | boolean, string?][];
 }
 interface TaskDeleteObject extends TaskBase {
  name: "deleteObject";
  objectName: string;
 }
 interface TaskProvisions extends TaskBase {
  name: "provisions";
  provisions?: [string, ...Value[]][];
 }
 export type Task =
  | TaskGetParameterValues
  | TaskSetParameterValues
  | TaskRefreshObject
  | TaskReboot
  | TaskFactoryReset
  | TaskDownload
  | TaskAddObject
  | TaskDeleteObject
  | TaskProvisions;
 export interface Operation {
  _id: string;
  name: string;
  timestamp: Date;
  channels: string;
  retries: string;
  provisions: string;
  args: string;
 }
 export interface Config {
  _id: string;
  value: string;
 }
 export interface Cache {
  _id: string;
  value: string;
  timestamp: Date;
  expire: Date;
 }
 export interface Device {
  _id: string;
  _lastInform: Date;
  _registered: Date;
  _tags?: string[];
  _timestamp?: Date;
 }
 type Configuration =
  | { type: "age"; name: string; age: number }
  | { type: "value"; name: string; value: boolean | number | string }
  | { type: "add_tag"; tag: string }
  | { type: "delete_tag"; tag: string }
  | { type: "add_object"; name: string; object: string }
  | { type: "delete_object"; name: string; object: string }
  | { type: "provision"; name: string; args?: string[] };
 export interface Preset {
  _id: string;
  weight: number;
  channel: string;
  events: Record<string, boolean>;
  configurations: Configuration[];
 }
 export interface Object {
  _id: string;
 }
 export interface Provision {
  _id: string;
  script: string;
 }
 export interface VirtualParameter {
  _id: string;
  script: string;
 }
 export interface File {
  _id: string;
  length: number;
  filename: string;
  uploadDate: Date;
  metadata?: {
    fileType?: string;
    oui?: string;
    productClass?: string;
    version?: string;
  };
 }
 export interface Permission {
  _id: string;
  role: string;
  resource: string;
  access: 1 | 2 | 3;
  filter?: string;
  validate?: string;
 }
 export interface User {
  _id: string;
  password: string;
  roles: string;
  salt: string;
 }
 export interface Lock {
  _id: string;
  value: string;
  timestamp: Date;
  expire: Date;
 }
@@ -1,183 +0,0 @@
 import Expression, { Value } from "../common/expression.ts";
 import Path from "../common/path.ts";
 import { encodeTag } from "../util.ts";
 // Optimize projection by removing overlaps
 // This can modify the object
 export function optimizeProjection(obj: { [path: string]: 1 }): {
  [path: string]: 1;
 } {
  if (obj[""]) return { "": obj[""] };
  const keys = Object.keys(obj).sort();
  if (keys.length <= 1) return obj;
  for (let i = 1; i < keys.length; ++i) {
    const a = keys[i - 1];
    const b = keys[i];
    if (b.startsWith(a)) {
      if (b.charAt(a.length) === "." || b.charAt(a.length - 1) === ".") {
        delete obj[b];
        keys.splice(i--, 1);
      }
    }
  }
  return obj;
 }
 export function convertOldPrecondition(q: Record<string, unknown>): Expression {
  function recursive(_query): Expression {
    let res: Expression = new Expression.Literal(true);
    for (const [k, v] of Object.entries(_query)) {
      if (k[0] === "$") {
        if (k === "$and") {
          for (const vv of Object.values(v))
            res = Expression.and(res, recursive(vv));
        } else if (k === "$or") {
          let or: Expression = new Expression.Literal(false);
          for (const vv of Object.values(v))
            or = Expression.or(or, recursive(vv));
          res = Expression.and(res, or);
        } else {
          throw new Error(`Operator ${k} not supported`);
        }
      } else if (k === "_tags") {
        if (typeof v === "object") {
          if (Array.isArray(v)) throw new Error(`Invalid type`);
          for (const [op, val] of Object.entries(v)) {
            if (op === "$ne") {
              if (typeof v["$ne"] !== "string")
                throw new Error("Only string values are allowed for _tags");
              res = Expression.and(
                res,
                new Expression.Unary(
                  "IS NULL",
                  new Expression.Parameter(
                    Path.parse(`Tags.${encodeTag(val)}`),
                  ),
                ),
              );
            } else if (op === "$eq") {
              if (typeof v["$eq"] !== "string")
                throw new Error("Only string values are allowed for _tags");
              res = Expression.and(
                res,
                new Expression.Unary(
                  "IS NOT NULL",
                  new Expression.Parameter(
                    Path.parse(`Tags.${encodeTag(val)}`),
                  ),
                ),
              );
            } else {
              throw new Error(`Invalid tag query`);
            }
          }
        } else {
          res = Expression.and(
            res,
            new Expression.Unary(
              "IS NOT NULL",
              new Expression.Parameter(
                Path.parse(`Tags.${encodeTag(v as string)}`),
              ),
            ),
          );
        }
      } else if (k.startsWith("Tags.")) {
        let exists: boolean;
        if (typeof v === "boolean") exists = v;
        else if (v.hasOwnProperty("$eq")) exists = !!v["$eq"];
        else if (v.hasOwnProperty("$ne")) exists = !v["$ne"];
        else if (v.hasOwnProperty("$exists")) exists = !!v["$exists"];
        else throw new Error(`Invalid tag query`);
        res = Expression.and(
          res,
          new Expression.Unary(
            exists ? "IS NOT NULL" : "IS NULL",
            new Expression.Parameter(Path.parse(k)),
          ),
        );
      } else if (typeof v === "object") {
        if (Array.isArray(v)) throw new Error(`Invalid type`);
        for (const [kk, vv] of Object.entries(v)) {
          if (kk === "$eq") {
            res = Expression.and(
              res,
              new Expression.Binary(
                "=",
                new Expression.Parameter(Path.parse(k)),
                new Expression.Literal(vv),
              ),
            );
          } else if (kk === "$ne") {
            const p = new Expression.Parameter(Path.parse(k));
            res = Expression.and(
              res,
              Expression.or(
                new Expression.Binary("<>", p, new Expression.Literal(vv)),
                new Expression.Unary("IS NULL", p),
              ),
            );
          } else if (kk === "$lt") {
            res = Expression.and(
              res,
              new Expression.Binary(
                "<",
                new Expression.Parameter(Path.parse(k)),
                new Expression.Literal(vv),
              ),
            );
          } else if (kk === "$lte") {
            res = Expression.and(
              res,
              new Expression.Binary(
                "<=",
                new Expression.Parameter(Path.parse(k)),
                new Expression.Literal(vv),
              ),
            );
          } else if (kk === "$gt") {
            res = Expression.and(
              res,
              new Expression.Binary(
                ">",
                new Expression.Parameter(Path.parse(k)),
                new Expression.Literal(vv),
              ),
            );
          } else if (kk === "$gte") {
            res = Expression.and(
              res,
              new Expression.Binary(
                ">=",
                new Expression.Parameter(Path.parse(k)),
                new Expression.Literal(vv),
              ),
            );
          } else {
            throw new Error(`Operator ${kk} not supported`);
          }
          if (!["string", "number", "boolean"].includes(typeof vv))
            throw new Error(`Invalid value for ${kk} operator`);
        }
      } else {
        res = Expression.and(
          res,
          new Expression.Binary(
            "=",
            new Expression.Parameter(Path.parse(k)),
            new Expression.Literal(v as Value),
          ),
        );
      }
    }
    return res;
  }
  // empty filter
  if (!Object.keys(q).length) return new Expression.Literal(true);
  return recursive(q);
 }
@@ -1,236 +0,0 @@
 import { IncomingMessage, ServerResponse, ClientRequest } from "node:http";
 import { Socket } from "node:net";
 import { appendFileSync } from "node:fs";
 import { stringify } from "./common/yaml.ts";
 import * as config from "./config.ts";
 import { getSocketEndpoints } from "./server.ts";
 const DEBUG_FILE = "" + config.get("DEBUG_FILE");
 const DEBUG_FORMAT = "" + config.get("DEBUG_FORMAT");
 const connectionTimestamps = new WeakMap<Socket, Date>();
 function getConnectionTimestamp(connection: Socket): Date {
  let t = connectionTimestamps.get(connection);
  if (!t) {
    t = new Date();
    connectionTimestamps.set(connection, t);
  }
  return t;
 }
 export function incomingHttpRequest(
  httpRequest: IncomingMessage,
  deviceId: string,
  body: string,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const con = httpRequest.socket;
  const socketEndpoints = getSocketEndpoints(con);
  const msg = {
    event: "incoming HTTP request",
    timestamp: now,
    remoteAddress: socketEndpoints.remoteAddress,
    deviceId: deviceId,
    connection: getConnectionTimestamp(con),
    localPort: socketEndpoints.localPort,
    method: httpRequest.method,
    url: httpRequest.url,
    headers: httpRequest.headers,
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function outgoingHttpResponse(
  httpResponse: ServerResponse,
  deviceId: string,
  body: string,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const con = httpResponse.socket;
  const socketEndpoints = getSocketEndpoints(con);
  const msg = {
    event: "outgoing HTTP response",
    timestamp: now,
    remoteAddress: socketEndpoints.remoteAddress,
    deviceId: deviceId,
    connection: getConnectionTimestamp(con),
    statusCode: httpResponse.statusCode,
    headers: httpResponse.getHeaders(),
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function outgoingHttpRequest(
  httpRequest: ClientRequest,
  deviceId: string,
  method: "GET" | "PUT" | "POST" | "DELETE",
  url: URL,
  body: string,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const con = httpRequest.socket;
  const msg = {
    event: "outgoing HTTP request",
    timestamp: now,
    remoteAddress: con.remoteAddress,
    deviceId: deviceId,
    connection: getConnectionTimestamp(con),
    remotePort: url.port,
    method: method,
    url: url.pathname + url.search,
    headers: httpRequest.getHeaders(),
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function outgoingHttpRequestError(
  httpRequest: ClientRequest,
  deviceId: string,
  method: "GET" | "PUT" | "POST" | "DELETE",
  url: URL,
  err: Error,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const msg = {
    event: "outgoing HTTP request",
    timestamp: now,
    remoteAddress: url.hostname,
    deviceId: deviceId,
    connection: null,
    remotePort: url.port,
    method: method,
    url: url.pathname + url.search,
    headers: httpRequest.getHeaders(),
    error: err.message,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function incomingHttpResponse(
  httpResponse: IncomingMessage,
  deviceId: string,
  body: string,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const con = httpResponse.socket;
  const msg = {
    event: "incoming HTTP response",
    timestamp: now,
    remoteAddress: con.remoteAddress,
    deviceId: deviceId,
    connection: getConnectionTimestamp(httpResponse.socket),
    statusCode: httpResponse.statusCode,
    headers: httpResponse.headers,
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function outgoingUdpMessage(
  remoteAddress: string,
  deviceId: string,
  remotePort: number,
  body: string,
 ): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const msg = {
    event: "outgoing UDP message",
    timestamp: now,
    remoteAddress: remoteAddress,
    deviceId: deviceId,
    remotePort: remotePort,
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function clientError(remoteAddress: string, err: Error): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const msg = {
    event: "client error",
    timestamp: now,
    remoteAddress: remoteAddress,
    error: err.message,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function outgoingXmppStanza(deviceId: string, body: string): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const msg = {
    event: "outgoing XMPP stanza",
    timestamp: now,
    deviceId: deviceId,
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
 export function incomingXmppStanza(deviceId: string, body: string): void {
  if (!DEBUG_FILE) return;
  const now = new Date();
  const msg = {
    event: "incoming XMPP stanza",
    timestamp: now,
    deviceId: deviceId,
    body: body,
  };
  if (DEBUG_FORMAT === "yaml")
    appendFileSync(DEBUG_FILE, "---\n" + stringify(msg));
  else if (DEBUG_FORMAT === "json")
    appendFileSync(DEBUG_FILE, JSON.stringify(msg) + "\n");
  else throw new Error(`Unrecognized DEBUG_FORMAT option`);
 }
@@ -1,259 +0,0 @@
 import Path from "./common/path.ts";
 import * as config from "./config.ts";
 import * as device from "./device.ts";
 import * as scheduling from "./scheduling.ts";
 import { SessionContext, Declaration } from "./types.ts";
 const MAX_DEPTH = +config.get("MAX_DEPTH");
 export function refresh(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (
    (provision.length !== 2 || typeof provision[1] !== "string") &&
    (provision.length !== 3 ||
      typeof provision[1] !== "string" ||
      typeof provision[2] !== "number") &&
    (provision.length < 4 ||
      typeof provision[1] !== "string" ||
      typeof provision[2] !== "number" ||
      typeof provision[3] !== "boolean")
  )
    throw new Error("Invalid arguments");
  const every = 1000 * ((provision[2] as number) || 1);
  const offset = scheduling.variance(sessionContext.deviceId, every);
  const t = scheduling.interval(sessionContext.timestamp, every, offset);
  let attrGet;
  let refreshChildren;
  if (provision[3] == null) {
    refreshChildren = true;
    attrGet = { object: 1, writable: 1, value: t };
  } else {
    attrGet = {};
    refreshChildren = !!provision[3];
    for (const a of provision.slice(4)) attrGet[a as string] = t;
  }
  let path = Path.parse(provision[1]);
  let l = path.length;
  if (refreshChildren) {
    const segments = path.segments.slice();
    l = segments.length;
    segments.length = MAX_DEPTH;
    segments.fill("*", l);
    path = Path.parse(segments.join("."));
  }
  for (let i = l; i <= path.length; ++i) {
    declarations.push({
      path: path.slice(0, i),
      pathGet: t,
      pathSet: null,
      attrGet: attrGet,
      attrSet: null,
      defer: true,
    });
  }
  return true;
 }
 export function value(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (
    provision.length < 3 ||
    provision.length > 4 ||
    typeof provision[1] !== "string"
  )
    throw new Error("Invalid arguments");
  let attr: string, val: any;
  if (provision.length === 3) {
    attr = "value";
    val = provision[2];
  } else {
    attr = (provision[2] as string) || "";
    val = provision[3];
  }
  if (attr === "accessList") {
    val = (val || "")
      .split(",")
      .map((s) => s.trim())
      .filter((s) => !!s);
  } else if (attr === "value") {
    val = [val];
  }
  declarations.push({
    path: Path.parse(provision[1]),
    pathGet: 1,
    pathSet: null,
    attrGet: { [attr]: 1 },
    attrSet: { [attr]: val },
    defer: true,
  });
  return true;
 }
 export function tag(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (
    provision.length !== 3 ||
    typeof provision[1] !== "string" ||
    typeof provision[2] !== "boolean"
  )
    throw new Error("Invalid arguments");
  declarations.push({
    path: Path.parse(`Tags.${provision[1]}`),
    pathGet: 1,
    pathSet: null,
    attrGet: { value: 1 },
    attrSet: { value: [provision[2]] },
    defer: true,
  });
  return true;
 }
 export function reboot(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (provision.length !== 1) throw new Error("Invalid arguments");
  declarations.push({
    path: Path.parse("Reboot"),
    pathGet: 1,
    pathSet: null,
    attrGet: { value: 1 },
    attrSet: { value: [sessionContext.timestamp] },
    defer: true,
  });
  return true;
 }
 export function reset(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (provision.length !== 1) throw new Error("Invalid arguments");
  declarations.push({
    path: Path.parse("FactoryReset"),
    pathGet: 1,
    pathSet: null,
    attrGet: { value: 1 },
    attrSet: { value: [sessionContext.timestamp] },
    defer: true,
  });
  return true;
 }
 export function download(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
 ): boolean {
  if (
    (provision.length !== 3 ||
      typeof provision[1] !== "string" ||
      typeof provision[2] !== "string") &&
    (provision.length !== 4 ||
      typeof provision[1] !== "string" ||
      typeof provision[2] !== "string" ||
      typeof provision[3] !== "string")
  )
    throw new Error("Invalid arguments");
  const alias = [
    `FileType:${JSON.stringify(provision[1] || "")}`,
    `FileName:${JSON.stringify(provision[2] || "")}`,
    `TargetFileName:${JSON.stringify(provision[3] || "")}`,
  ].join(",");
  declarations.push({
    path: Path.parse(`Downloads.[${alias}]`),
    pathGet: 1,
    pathSet: 1,
    attrGet: null,
    attrSet: null,
    defer: true,
  });
  declarations.push({
    path: Path.parse(`Downloads.[${alias}].Download`),
    pathGet: 1,
    pathSet: null,
    attrGet: { value: 1 },
    attrSet: { value: [sessionContext.timestamp] },
    defer: true,
  });
  return true;
 }
 export function instances(
  sessionContext: SessionContext,
  provision: (string | number | boolean)[],
  declarations: Declaration[],
  startRevision: number,
  endRevision: number,
 ): boolean {
  if (provision.length !== 3 || typeof provision[1] !== "string")
    throw new Error("Invalid arguments");
  let count = Number(provision[2]);
  if (Number.isNaN(count)) throw new Error("Invalid arguments");
  const path = Path.parse(provision[1]);
  if (provision[2][0] === "+" || provision[2][0] === "-") {
    declarations.push({
      path: path,
      pathGet: 1,
      pathSet: null,
      attrGet: null,
      attrSet: null,
      defer: true,
    });
    if (endRevision === startRevision) return false;
    const unpacked = device.unpack(
      sessionContext.deviceData,
      path,
      startRevision + 1,
    );
    count = Math.max(0, unpacked.length + count);
  }
  declarations.push({
    path: path,
    pathGet: 1,
    pathSet: count,
    attrGet: null,
    attrSet: null,
    defer: true,
  });
  return true;
 }
@@ -1,419 +0,0 @@
 import Expression, { Value } from "./common/expression.ts";
 import Path from "./common/path.ts";
 import {
  DeviceData,
  Declaration,
  Clear,
  AttributeTimestamps,
  Attributes,
 } from "./types.ts";
 const CHANGE_FLAGS = {
  object: 2,
  writable: 4,
  value: 8,
  notification: 16,
  accessList: 32,
 };
 function parseBool(v): boolean {
  v = "" + v;
  if (v === "true" || v === "TRUE" || v === "True" || v === "1") return true;
  else if (v === "false" || v === "FALSE" || v === "False" || v === "0")
    return false;
  else return null;
 }
 export function sanitizeParameterValue(
  parameterValue: [string | number | boolean, string],
 ): [string | number | boolean, string] {
  if (parameterValue[0] != null) {
    switch (parameterValue[1]) {
      case "xsd:boolean":
        if (typeof parameterValue[0] !== "boolean") {
          const b = parseBool(parameterValue[0]);
          if (b == null) parameterValue[0] = "" + parameterValue[0];
          else parameterValue[0] = b;
        }
        break;
      case "xsd:int":
      case "xsd:unsignedInt":
        if (typeof parameterValue[0] !== "number") {
          const i = parseInt(parameterValue[0] as string);
          if (isNaN(i)) parameterValue[0] = "" + parameterValue[0];
          else parameterValue[0] = i;
        }
        break;
      case "xsd:dateTime":
        if (typeof parameterValue[0] !== "number") {
          // Don't use parseInt because it reads date string as a number
          let i = +parameterValue[0];
          if (isNaN(i)) {
            i = Date.parse(parameterValue[0] as string);
            if (isNaN(i)) parameterValue[0] = "" + parameterValue[0];
            else parameterValue[0] = i;
          } else {
            parameterValue[0] = i;
          }
        }
        break;
      default:
        parameterValue[0] = "" + parameterValue[0];
        break;
    }
  }
  return parameterValue;
 }
 export function getAliasDeclarations(
  path: Path,
  timestamp: number,
  attrGet = null,
 ): Declaration[] {
  const stripped = path.stripAlias();
  let decs: Declaration[] = [
    {
      path: stripped,
      pathGet: timestamp,
      pathSet: null,
      attrGet: attrGet,
      attrSet: null,
      defer: true,
    },
  ];
  if (path.alias) {
    for (const [i, alias] of path.segments.entries()) {
      if (alias instanceof Expression) {
        const parent = stripped.slice(0, i + 1);
        for (const [p] of expressionToAlias(alias)) {
          decs = decs.concat(
            getAliasDeclarations(parent.concat(p), timestamp, {
              value: timestamp,
            }),
          );
        }
      }
    }
  }
  return decs;
 }
 export function expressionToAlias(exp: Expression): [Path, Value][] {
  if (exp instanceof Expression.Literal && exp.value === true) return [];
  if (exp instanceof Expression.Binary) {
    if (exp.operator === "AND")
      return [...expressionToAlias(exp.left), ...expressionToAlias(exp.right)];
    else if (exp.operator === "=") {
      if (
        exp.left instanceof Expression.Parameter &&
        exp.right instanceof Expression.Literal
      )
        return [[exp.left.path, exp.right.value]];
    }
  }
  throw new Error("Invalid alias expression");
 }
 export function unpack(
  deviceData: DeviceData,
  path: Path,
  revision?: number,
 ): Path[] {
  let allMatches = [] as Path[];
  if (!path.alias) {
    for (const p of deviceData.paths.findCompat(path, false, true))
      if (deviceData.attributes.has(p, revision)) allMatches.push(p);
  } else {
    const wildcardPath = path.stripAlias();
    for (const p of deviceData.paths.findCompat(wildcardPath, false, true))
      if (deviceData.attributes.has(p, revision)) allMatches.push(p);
    for (let i = path.length - 1; i >= 0; --i) {
      if (path.alias & (1 << i)) {
        for (const [param, val] of expressionToAlias(
          path.segments[i] as Expression,
        )) {
          const p = wildcardPath.slice(0, i + 1).concat(param);
          const unpacked = unpack(deviceData, p, revision);
          const filtered: Path[] = [];
          for (const up of unpacked) {
            const attributes = deviceData.attributes.get(up, revision);
            if (
              attributes &&
              attributes.value &&
              attributes.value[1] &&
              sanitizeParameterValue([val, attributes.value[1][1]])[0] ===
                attributes.value[1][0]
            ) {
              for (let m = 0; m < allMatches.length; ++m) {
                let k;
                const match = allMatches[m];
                if (!match) continue;
                for (k = i; k >= 0; --k)
                  if (match.segments[k] !== up.segments[k]) break;
                if (k < 0) {
                  filtered.push(match);
                  allMatches[m] = null;
                }
              }
            }
          }
          allMatches = filtered;
        }
      }
    }
  }
  allMatches.sort((p1, p2) => {
    for (let i = 0; i < p1.length; ++i) {
      const a = p1.segments[i] as string;
      const b = p2.segments[i] as string;
      if (a !== b) {
        // Use numeric sorting for numbers
        const ia = parseInt(a);
        const ib = parseInt(b);
        if (ia === +a && ib === +b) return ia - ib;
        else if (a < b) return -1;
        else return 1;
      }
    }
    return 0;
  });
  return allMatches;
 }
 export function clear(
  deviceData: DeviceData,
  path: Path,
  timestamp: number,
  attributes: AttributeTimestamps,
  changeFlags = 0,
 ): void {
  const changeTrackers = {};
  timestamp = timestamp || 0;
  let descendantsTimestamp = timestamp;
  if (attributes?.object) {
    if (attributes.object > descendantsTimestamp)
      descendantsTimestamp = attributes.object;
    if (!(attributes.object <= attributes.value))
      attributes.value = attributes.object;
  }
  for (const p of deviceData.paths.findCompat(
    path,
    true,
    true,
    descendantsTimestamp ? 99 : path.length,
  )) {
    const tracker = deviceData.trackers.get(p);
    for (const k in tracker) changeTrackers[k] |= tracker[k];
    const currentTimestamp = deviceData.timestamps.get(p);
    if (currentTimestamp === undefined) continue;
    if (
      timestamp > currentTimestamp ||
      (descendantsTimestamp > currentTimestamp && p.length > path.length)
    ) {
      deviceData.timestamps.delete(p);
      deviceData.attributes.delete(p);
      changeFlags |= 1;
    } else if (attributes && p.length === path.length) {
      const currentAttributes = deviceData.attributes.get(p);
      if (currentAttributes) {
        let newAttrs;
        for (const attrName in attributes) {
          if (
            attrName in currentAttributes &&
            attributes[attrName] > currentAttributes[attrName][0]
          ) {
            changeFlags |= CHANGE_FLAGS[attrName];
            if (!newAttrs) {
              newAttrs = Object.assign({}, currentAttributes);
              deviceData.attributes.set(p, newAttrs);
            }
            delete newAttrs[attrName];
          }
        }
      }
    }
  }
  // Note: For performance, we're merging all changes together rather than
  // mark changes based the exact parameters affected.
  for (const k in changeTrackers)
    if (changeTrackers[k] & changeFlags) deviceData.changes.add(k);
 }
 function compareEquality(a, b): boolean {
  const t = typeof a;
  if (
    a === null ||
    a === undefined ||
    t === "number" ||
    t === "boolean" ||
    t === "string" ||
    t === "symbol"
  )
    return a === b;
  return JSON.stringify(a) === JSON.stringify(b);
 }
 export function set(
  deviceData: DeviceData,
  pathStr: string,
  timestamp: number,
  attributes: Attributes,
  toClear?: Clear[],
 ): Clear[] {
  const path = deviceData.paths.add(pathStr);
  const currentTimestamp = deviceData.timestamps.get(path);
  let currentAttributes;
  if (path.wildcard) attributes = undefined;
  else if (currentTimestamp)
    currentAttributes = deviceData.attributes.get(path);
  let changeFlags = 0;
  if (attributes) {
    if (
      attributes.value &&
      attributes.value[1] &&
      attributes.value[0] >= (attributes.object ? attributes.object[0] : 0)
    )
      attributes.object = [attributes.value[0], 0];
    if (
      attributes.object &&
      attributes.object[1] &&
      attributes.object[0] >= (attributes.value ? attributes.value[0] : 0)
    )
      attributes.value = [attributes.object[0], null];
    const newAttributes = Object.assign({}, currentAttributes, attributes);
    if (currentAttributes) {
      for (const attrName in attributes) {
        timestamp = Math.max(timestamp, attributes[attrName][0]);
        if (!(attrName in currentAttributes))
          changeFlags |= CHANGE_FLAGS[attrName];
        else if (attributes[attrName][0] <= currentAttributes[attrName][0])
          newAttributes[attrName] = currentAttributes[attrName];
        else if (
          !compareEquality(
            attributes[attrName][1],
            currentAttributes[attrName][1],
          )
        )
          changeFlags |= CHANGE_FLAGS[attrName];
      }
    } else {
      changeFlags |= 1;
    }
    deviceData.attributes.set(path, newAttributes);
    if (!(timestamp <= currentTimestamp)) {
      deviceData.timestamps.set(path, timestamp);
      if (path.length > 1) {
        toClear = set(
          deviceData,
          path.slice(0, path.length - 1).toString(),
          timestamp,
          { object: [timestamp, 1] },
          toClear,
        );
      }
    }
  } else if (!(timestamp <= currentTimestamp)) {
    deviceData.timestamps.set(path, timestamp);
    if (currentAttributes) {
      deviceData.attributes.delete(path);
      changeFlags |= 1;
    } else if (path.wildcard) {
      for (const p of deviceData.paths.findCompat(
        path,
        false,
        true,
        path.length,
      )) {
        if (timestamp > deviceData.timestamps.get(p)) {
          toClear = toClear || [];
          toClear.push([p, timestamp]);
        }
      }
    }
  }
  if (changeFlags) {
    if (changeFlags & 1) {
      toClear = toClear || [];
      toClear.push([path, timestamp, null, changeFlags]);
    } else if (changeFlags & CHANGE_FLAGS.object) {
      toClear = toClear || [];
      toClear.push([path, 0, { object: attributes.object[0] }, changeFlags]);
    } else {
      for (const p of deviceData.paths.findCompat(
        path,
        true,
        false,
        path.length,
      )) {
        const tracker = deviceData.trackers.get(p);
        for (const k in tracker)
          if (tracker[k] & changeFlags) deviceData.changes.add(k);
      }
    }
  }
  return toClear;
 }
 export function track(
  deviceData: DeviceData,
  pathStr: string,
  marker: string,
  attributes?: string[],
 ): void {
  const path = deviceData.paths.add(pathStr);
  let f = 1;
  if (attributes)
    for (const attrName of attributes) f |= CHANGE_FLAGS[attrName];
  let cur = deviceData.trackers.get(path);
  if (!cur) {
    cur = {};
    deviceData.trackers.set(path, cur);
  }
  cur[marker] |= f;
 }
 export function clearTrackers(
  deviceData: DeviceData,
  tracker: string | string[],
 ): void {
  if (Array.isArray(tracker)) {
    for (const v of deviceData.trackers.values())
      for (const t of tracker) delete v[t];
    for (const t of tracker) deviceData.changes.delete(t);
  } else {
    for (const v of deviceData.trackers.values()) delete v[tracker];
    deviceData.changes.delete(tracker);
  }
 }
@@ -1,109 +0,0 @@
 import { spawn, ChildProcess } from "node:child_process";
 import * as crypto from "node:crypto";
 import readline from "node:readline";
 import * as config from "./config.ts";
 import { Fault } from "./types.ts";
 import { ROOT_DIR } from "./config.ts";
 import * as logger from "./logger.ts";
 const TIMEOUT = +config.get("EXT_TIMEOUT");
 const processes: { [script: string]: ChildProcess } = {};
 const jobs = new Map();
 export function run(args: string[]): Promise<{ fault: Fault; value: any }> {
  return new Promise((resolve) => {
    const scriptName = args[0];
    const id = crypto.randomBytes(8).toString("hex");
    jobs.set(id, resolve);
    if (!processes[scriptName]) {
      const p = spawn(ROOT_DIR + "/bin/genieacs-ext", [scriptName], {
        stdio: ["ignore", "pipe", "pipe", "ipc"],
      });
      processes[scriptName] = p;
      p.on("error", (err) => {
        if (processes[scriptName] === p) {
          if (jobs.delete(id)) {
            resolve({
              fault: { code: err.name, message: err.message },
              value: null,
            });
          }
          // eslint-disable-next-line @typescript-eslint/no-floating-promises
          kill(processes[scriptName]);
          delete processes[scriptName];
        }
      });
      p.on("disconnect", () => {
        if (processes[scriptName] === p) delete processes[scriptName];
      });
      p.on("message", (message) => {
        const func = jobs.get(message[0]);
        if (func) {
          jobs.delete(message[0]);
          // Wait for any disconnect even to fire
          setTimeout(() => {
            func({ fault: message[1], value: message[2] });
          });
        }
      });
      const rlstdout = readline.createInterface(p.stdout);
      rlstdout.on("line", (line) => {
        logger.info({ message: `Ext ${scriptName}(${p.pid}): ${line}` });
      });
      const rlstderr = readline.createInterface(p.stderr);
      rlstderr.on("line", (line) => {
        logger.warn({ message: `Ext ${scriptName}(${p.pid}): ${line}` });
      });
    }
    setTimeout(() => {
      if (jobs.delete(id)) {
        resolve({
          fault: { code: "timeout", message: "Extension timed out" },
          value: null,
        });
      }
    }, TIMEOUT);
    if (!processes[scriptName].connected) return false;
    return processes[scriptName].send([id, args.slice(1)]);
  });
 }
 function kill(process: ChildProcess): Promise<void> {
  return new Promise((resolve) => {
    const timeToKill = Date.now() + 5000;
    process.kill();
    const t = setInterval(() => {
      if (!process.connected) {
        clearInterval(t);
        resolve();
      } else if (Date.now() > timeToKill) {
        process.kill("SIGKILL");
        clearInterval(t);
        resolve();
      }
    }, 100);
  });
 }
 export async function killAll(): Promise<void> {
  await Promise.all(
    Object.entries(processes).map(([k, p]) => {
      delete processes[k];
      return kill(p);
    }),
  );
 }
@@ -1,160 +0,0 @@
 import { IncomingMessage } from "node:http";
 import { TLSSocket } from "node:tls";
 import { parseCIDR, parse, IPv6, IPv4 } from "ipaddr.js";
 import * as config from "./config.ts";
 import { getSocketEndpoints } from "./server.ts";
 interface RequestOrigin {
  localAddress: string;
  localPort: number;
  remoteAddress: string;
  remotePort: number;
  host: string;
  encrypted: boolean;
 }
 const FORWARDED_HEADER = "" + config.get("FORWARDED_HEADER");
 const cache = new WeakMap<IncomingMessage, RequestOrigin>();
 const cidrs: [IPv4 | IPv6, number][] = [];
 for (const str of FORWARDED_HEADER.split(",").map((s) => s.trim())) {
  try {
    cidrs.push(parseCIDR(str));
  } catch {
    // Not a valid CIDR format, try parsing as IP
    try {
      const ip = parse(str);
      cidrs.push([ip, ip.toByteArray().length * 8]);
    } catch {
      // Not a valid IP either, ignore
    }
  }
 }
 function parseForwardedHeader(str: string): { [name: string]: string } {
  str = str.toLowerCase();
  const res: { [name: string]: string } = {};
  let keyIdx = 0;
  let valueIdx = -1;
  let key: string;
  for (let i = 0; i < str.length; ++i) {
    const char = str.charCodeAt(i);
    if (char === 61 /* = */) {
      if (keyIdx >= 0) {
        key = str.slice(keyIdx, i).trim();
        keyIdx = -1;
        valueIdx = i + 1;
      }
    } else if (char === 59 /* ; */) {
      if (valueIdx >= 0) res[key] = str.slice(valueIdx, i).trim();
      valueIdx = -1;
      keyIdx = i + 1;
    } else if (char === 44 /* , */) {
      if (valueIdx >= 0) res[key] = str.slice(valueIdx, i).trim();
      return res;
    } else if (char === 34 /* " */) {
      if (valueIdx >= 0) {
        const quoteIdx = i;
        if (!str.slice(valueIdx, quoteIdx).trim()) {
          for (i = i + 1; i < str.length; ++i) {
            const c = str.charCodeAt(i);
            if (c === 92 /* \ */) ++i;
            if (c === 34 /* " */) {
              res[key] = JSON.parse(str.slice(quoteIdx, i + 1).trim());
              valueIdx = -1;
              keyIdx = i + 1;
              break;
            }
          }
        }
      }
    }
  }
  if (valueIdx >= 0) res[key] = str.slice(valueIdx).trim();
  return res;
 }
 export function getRequestOrigin(request: IncomingMessage): RequestOrigin {
  let origin = cache.get(request);
  if (!origin) {
    const soc = request.socket;
    const socketEndpoints = getSocketEndpoints(soc);
    origin = {
      localAddress: socketEndpoints.localAddress,
      localPort: socketEndpoints.localPort,
      remoteAddress: socketEndpoints.remoteAddress,
      remotePort: socketEndpoints.remotePort,
      host: request.headers["host"],
      encrypted: !!(request.socket as TLSSocket).encrypted,
    };
    const header = request.headers["forwarded"];
    if (header) {
      const ip = parse(socketEndpoints.remoteAddress);
      if (
        cidrs.some((cidr) => ip.kind() === cidr[0].kind() && ip.match(cidr))
      ) {
        const parsed = parseForwardedHeader(header);
        if (parsed["proto"] === "https") {
          origin.encrypted = true;
          origin.localPort = 443;
        } else if (parsed["proto"] === "http") {
          origin.encrypted = false;
          origin.localPort = 80;
        }
        if (parsed["host"]) {
          origin.host = parsed["host"];
          const [, port] = parsed["host"].split(":", 2);
          origin.localPort = +port || origin.localPort;
        }
        if (parsed["for"]) {
          if (parsed["for"].startsWith("[")) {
            const i = parsed["for"].lastIndexOf("]");
            if (i >= 0) {
              origin.remoteAddress = parsed["for"].slice(1, i);
              origin.remotePort =
                parseInt(parsed["for"].slice(i + 2)) || origin.remotePort;
            }
          } else {
            const i = parsed["for"].lastIndexOf(":");
            if (i >= 0) {
              origin.remoteAddress = parsed["for"].slice(0, i);
              origin.remotePort =
                parseInt(parsed["for"].slice(i + 1)) || origin.remotePort;
            } else {
              origin.remoteAddress = parsed["for"];
            }
          }
        }
        if (parsed["by"]) {
          if (parsed["by"].startsWith("[")) {
            const i = parsed["by"].lastIndexOf("]");
            if (i >= 0) {
              origin.localAddress = parsed["by"].slice(1, i);
              origin.localPort =
                parseInt(parsed["by"].slice(i + 2)) || origin.localPort;
            }
          } else {
            const i = parsed["by"].lastIndexOf(":");
            if (i >= 0) {
              origin.localAddress = parsed["by"].slice(0, i);
              origin.localPort =
                parseInt(parsed["by"].slice(i + 1)) || origin.localPort;
            } else {
              origin.localAddress = parsed["by"];
            }
          }
        }
      }
    }
    cache.set(request, origin);
  }
  return origin;
 }
@@ -0,0 +1,49 @@
 ###
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 config = require './config'
 url = require 'url'
 mongodb = require 'mongodb'
 querystring = require 'querystring'
 db = require './db'
 listener = (request, response) ->
  urlParts = url.parse(request.url, true)
  if request.method == 'GET'
    filename = querystring.unescape(urlParts.pathname.substring(1))
    gs = new mongodb.GridStore(db.mongoDb, filename, 'r', {})
    gs.open((err, gs) ->
      if err
        response.writeHead(404)
        response.end()
        return
      stream = gs.stream(true)
      response.writeHead(200, {'Content-Type' : 'application/octet-stream', 'Content-Length' : gs.length})
      stream.pipe(response)
    )
  else
    response.writeHead(405, {'Allow': 'GET'})
    response.end('405 Method Not Allowed')
 exports.listener = listener
@@ -1,203 +0,0 @@
 import * as url from "node:url";
 import { IncomingMessage, ServerResponse } from "node:http";
 import { PassThrough, pipeline, Readable } from "node:stream";
 import { createHash } from "node:crypto";
 import { filesBucket, collections } from "./db/db.ts";
 import * as logger from "./logger.ts";
 import { getRequestOrigin } from "./forwarded.ts";
 import memoize from "./common/memoize.ts";
 const getFile = memoize(
  async (
    etag: string,
    size: number,
    filename: string,
  ): Promise<Iterable<Buffer>> => {
    const chunks: Buffer[] = [];
    // Using for-await over the download stream can throw ERR_STREAM_PREMATURE_CLOSE
    // for very small files. Possibly a bug in MongoDB driver or Nodejs itself.
    // Using a PassThrough stream to avoid this.
    const downloadStream = pipeline(
      filesBucket.openDownloadStreamByName(filename),
      new PassThrough(),
      (err) => {
        if (err) throw err;
      },
    );
    for await (const chunk of downloadStream) chunks.push(chunk);
    // Node 12-14 don't throw error when stream is closed prematurely.
    // However, we don't need to check for that since we're checking file size.
    if (size !== chunks.reduce((a, b) => a + b.length, 0))
      throw new Error("File size mismatch");
    return chunks;
  },
 );
 async function* partialContent(
  chunks: Iterable<Buffer>,
  start: number,
  end: number,
 ): AsyncIterable<Buffer> {
  let bytesToSkip = start;
  let bytesToRead = end - start;
  for (let chunk of chunks) {
    if (bytesToRead <= 0) return;
    if (bytesToSkip >= chunk.length) {
      bytesToSkip -= chunk.length;
      continue;
    }
    chunk = chunk.subarray(bytesToSkip, bytesToSkip + bytesToRead);
    bytesToRead -= chunk.length;
    bytesToSkip = 0;
    yield chunk;
  }
 }
 function generateETag(file: {
  _id: string;
  uploadDate: Date;
  length: number;
 }): string {
  const hash = createHash("md5");
  hash.update(`${file._id}-${file.uploadDate.getTime()}-${file.length}`);
  return hash.digest("hex");
 }
 function matchEtag(etag: string, header: string): boolean {
  for (let t of header.split(",")) {
    t = t.trim();
    if (t.startsWith("W/")) t = t.substring(2);
    try {
      t = JSON.parse(t);
    } catch {
      // Ignore
    }
    if (t === "*") return true;
    if (etag === t) return true;
  }
  return false;
 }
 export async function listener(
  request: IncomingMessage,
  response: ServerResponse,
 ): Promise<void> {
  if (request.method !== "GET" && request.method !== "HEAD") {
    response.writeHead(405, { Allow: "GET, HEAD" });
    response.end("405 Method Not Allowed");
    return;
  }
  const urlParts = url.parse(request.url, true);
  const filename = decodeURIComponent(urlParts.pathname.substring(1));
  const log = {
    message: "Fetch file",
    filename: filename,
    remoteAddress: getRequestOrigin(request).remoteAddress,
    method: request.method,
  };
  const file = await collections.files.findOne({ _id: filename });
  if (!file) {
    response.writeHead(404);
    response.end();
    log.message += " not found";
    logger.accessError(log);
    return;
  }
  logger.accessInfo(log);
  const etag = generateETag(file);
  const lastModified = file["uploadDate"];
  lastModified.setMilliseconds(0);
  let status = 200;
  let start = 0;
  let end = file.length;
  if (request.headers["if-match"])
    if (!matchEtag(etag, request.headers["if-match"])) status = 412;
  if (request.headers["if-unmodified-since"]) {
    const d = new Date(request.headers["if-unmodified-since"]);
    if (lastModified > d) status = 412;
  }
  if (request.headers["if-none-match"]) {
    if (matchEtag(etag, request.headers["if-none-match"])) status = 304;
  } else if (request.headers["if-modified-since"]) {
    const d = new Date(request.headers["if-modified-since"]);
    if (lastModified <= d) status = 304;
  }
  if (request.headers.range && status === 200) {
    const match = request.headers.range.match(/^bytes=(\d*)-(\d*)$/);
    status = 416;
    if (match && (match[1] || match[2])) {
      if (match[2]) end = parseInt(match[2]) + 1;
      if (match[1]) start = parseInt(match[1]);
      else start = file.length - parseInt(match[2]);
      if (start < end && end <= file.length) status = 206;
    }
    if (request.headers["if-range"]) {
      const h = request.headers["if-range"] as string;
      const d = new Date(h);
      if (!matchEtag(etag, h) && !(lastModified <= d)) {
        status = 200;
        start = 0;
        end = file.length;
      }
    }
  }
  if (status === 412) {
    response.writeHead(412);
    response.end();
    return;
  }
  if (status === 304) {
    response.writeHead(304, {
      ETag: etag,
      "Last-Modified": lastModified.toUTCString(),
    });
    response.end();
    return;
  }
  if (status === 416) {
    response.writeHead(416, {
      "Content-Range": `bytes */${file.length}`,
      "Content-Length": "0",
    });
    response.end();
    return;
  }
  response.writeHead(status, {
    "Content-Type": "application/octet-stream",
    "Content-Length": end - start,
    "Accept-Ranges": "bytes",
    ETag: etag,
    "Last-Modified": lastModified.toUTCString(),
    ...(status === 206 && {
      "Content-Range": `bytes ${start}-${end - 1}/${file.length}`,
    }),
  });
  if (request.method === "HEAD") {
    response.end();
    return;
  }
  const chunks = await getFile(etag, file.length, filename);
  pipeline(Readable.from(partialContent(chunks, start, end)), response, () => {
    // Ignore errors resulting from client disconnecting
  });
 }
@@ -1,64 +0,0 @@
 import Path from "./common/path.ts";
 const WILDCARD_MULTIPLIER = 2;
 const UNDISCOVERED_DEPTH = 7;
 // Simple heuristic to estimate GPN count given a set of patterns to be
 // discovered. Used to decide whether to use nextLevel = false in GPN.
 // gpnPatterns is [pattern, flags]
 // pattern is a path (array)
 // flags is an int where its bits mark the segments in the pattern that
 // need refreshing. Leading 0s indicate that the pattern up to that
 // point has been discovered.
 export function estimateGpnCount(
  gpnPatterns: [Path, number][],
  depth = 0,
 ): number {
  const children: { [segment: string]: [Path, number][] } = {};
  const wildcardChildren: [Path, number][] = [];
  let wildcardDiscovered = false;
  let gpnCount = 0;
  for (const pattern of gpnPatterns) {
    const path = pattern[0];
    const flags = pattern[1] >> depth;
    const k = path.segments[depth] as string;
    if (!k) {
      if (flags & 1) gpnCount = 1;
      continue;
    }
    if (flags & 1) {
      gpnCount = 1;
      if (depth > UNDISCOVERED_DEPTH) continue;
    } else if (k === "*") {
      wildcardDiscovered = true;
    }
    if (k === "*") {
      wildcardChildren.push(pattern);
    } else {
      children[k] = children[k] || [];
      children[k].push(pattern);
    }
  }
  let wildcardGpnCount = 0;
  if (!wildcardDiscovered && wildcardChildren.length) {
    wildcardGpnCount +=
      estimateGpnCount(wildcardChildren, depth + 1) * WILDCARD_MULTIPLIER;
  }
  for (const k of Object.keys(children)) {
    const c = estimateGpnCount(children[k].concat(wildcardChildren), depth + 1);
    wildcardGpnCount -= c;
    gpnCount += c;
  }
  gpnCount += Math.max(0, wildcardGpnCount);
  return gpnCount;
 }
@@ -1,376 +0,0 @@
 import { getRevision, getUiConfig, getUsers } from "./ui/local-cache.ts";
 import { generateSalt, hashPassword } from "./auth.ts";
 import { collections } from "./db/db.ts";
 import {
  putConfig,
  putPermission,
  putPreset,
  putProvision,
  putUser,
 } from "./ui/db.ts";
 import { del } from "./cache.ts";
 import BOOTSTRAP_SCRIPT from "../seed/bootstrap.js" with { type: "text" };
 import DEFAULT_SCRIPT from "../seed/default.js" with { type: "text" };
 import INFORM_SCRIPT from "../seed/inform.js" with { type: "text" };
 interface Status {
  users: boolean;
  presets: boolean;
  filters: boolean;
  device: boolean;
  index: boolean;
  overview: boolean;
 }
 export async function getStatus(): Promise<Status> {
  const [configSnapshot, presetCount] = await Promise.all([
    getRevision(),
    collections.presets.countDocuments(),
  ]);
  const users = getUsers(configSnapshot);
  const ui = getUiConfig(configSnapshot);
  const status = {
    users: !Object.keys(users).length,
    presets: !presetCount,
    filters: true,
    device: true,
    index: true,
    overview: true,
  };
  for (const k of Object.keys(ui)) {
    if (k.startsWith("filters.")) status.filters = false;
    if (k.startsWith("device.")) status.device = false;
    if (k.startsWith("index.")) status.index = false;
    if (k.startsWith("overview.")) status.overview = false;
  }
  return status;
 }
 export async function seed(options: Record<string, boolean>): Promise<void> {
  const resources = {};
  const proms = [];
  if (options.users) {
    resources["permissions"] = [
      { role: "admin", resource: "devices", access: 3, validate: "true" },
      { role: "admin", resource: "faults", access: 3, validate: "true" },
      { role: "admin", resource: "files", access: 3, validate: "true" },
      { role: "admin", resource: "presets", access: 3, validate: "true" },
      { role: "admin", resource: "provisions", access: 3, validate: "true" },
      { role: "admin", resource: "config", access: 3, validate: "true" },
      { role: "admin", resource: "permissions", access: 3, validate: "true" },
      { role: "admin", resource: "users", access: 3, validate: "true" },
      {
        role: "admin",
        resource: "virtualParameters",
        access: 3,
        validate: "true",
      },
    ];
    resources["users"] = [
      { username: "admin", password: "admin", roles: ["admin"] },
    ];
  }
  if (options.filters) {
    resources["config"] = (resources["config"] || []).concat([
      { _id: "ui.filters.0.label", value: "'Serial number'" },
      { _id: "ui.filters.0.parameter", value: "DeviceID.SerialNumber" },
      { _id: "ui.filters.0.type", value: "'string'" },
      { _id: "ui.filters.1.label", value: "'Product class'" },
      { _id: "ui.filters.1.parameter", value: "DeviceID.ProductClass" },
      { _id: "ui.filters.1.type", value: "'string'" },
      { _id: "ui.filters.2.label", value: "'Tag'" },
      { _id: "ui.filters.2.type", value: "'tag'" },
    ]);
  }
  if (options.device) {
    resources["config"] = (resources["config"] || []).concat([
      { _id: "ui.device.0.type", value: "'tags'" },
      { _id: "ui.device.1.type", value: "'ping'" },
      { _id: "ui.device.2.type", value: "'parameter-list'" },
      { _id: "ui.device.2.parameters.0.type", value: "'container'" },
      { _id: "ui.device.2.parameters.0.element", value: "'span.inform'" },
      { _id: "ui.device.2.parameters.0.label", value: "'Last inform'" },
      {
        _id: "ui.device.2.parameters.0.components.0.type",
        value: "'parameter'",
      },
      { _id: "ui.device.2.parameters.0.components.1.chart", value: "'online'" },
      {
        _id: "ui.device.2.parameters.0.components.1.type",
        value: "'overview-dot'",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.type",
        value: "'summon-button'",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.0",
        value: "InternetGatewayDevice.DeviceInfo.HardwareVersion",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.1",
        value: "InternetGatewayDevice.DeviceInfo.SoftwareVersion",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.2",
        value:
          "InternetGatewayDevice.WANDevice.*.WANConnectionDevice.*.WANIPConnection.*.MACAddress",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.3",
        value:
          "InternetGatewayDevice.WANDevice.*.WANConnectionDevice.*.WANIPConnection.*.ExternalIPAddress",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.4",
        value: "InternetGatewayDevice.LANDevice.*.WLANConfiguration.*.SSID",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.5",
        value:
          "InternetGatewayDevice.LANDevice.*.WLANConfiguration.*.KeyPassphrase",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.6",
        value: "InternetGatewayDevice.LANDevice.*.Hosts.Host.*.HostName",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.7",
        value: "InternetGatewayDevice.LANDevice.*.Hosts.Host.*.IPAddress",
      },
      {
        _id: "ui.device.2.parameters.0.components.2.parameters.8",
        value: "InternetGatewayDevice.LANDevice.*.Hosts.Host.*.MACAddress",
      },
      {
        _id: "ui.device.2.parameters.0.parameter",
        value: "DATE_STRING(Events.Inform)",
      },
      { _id: "ui.device.2.parameters.1.label", value: "'Serial number'" },
      {
        _id: "ui.device.2.parameters.1.parameter",
        value: "DeviceID.SerialNumber",
      },
      { _id: "ui.device.2.parameters.2.label", value: "'Product class'" },
      {
        _id: "ui.device.2.parameters.2.parameter",
        value: "DeviceID.ProductClass",
      },
      { _id: "ui.device.2.parameters.3.label", value: "'OUI'" },
      {
        _id: "ui.device.2.parameters.3.parameter",
        value: "DeviceID.OUI",
      },
      { _id: "ui.device.2.parameters.4.label", value: "'Manufacturer'" },
      {
        _id: "ui.device.2.parameters.4.parameter",
        value: "DeviceID.Manufacturer",
      },
      { _id: "ui.device.2.parameters.5.label", value: "'Hardware version'" },
      {
        _id: "ui.device.2.parameters.5.parameter",
        value: "InternetGatewayDevice.DeviceInfo.HardwareVersion",
      },
      { _id: "ui.device.2.parameters.6.label", value: "'Software version'" },
      {
        _id: "ui.device.2.parameters.6.parameter",
        value: "InternetGatewayDevice.DeviceInfo.SoftwareVersion",
      },
      { _id: "ui.device.2.parameters.7.label", value: "'MAC'" },
      {
        _id: "ui.device.2.parameters.7.parameter",
        value:
          "InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.1.MACAddress",
      },
      { _id: "ui.device.2.parameters.8.label", value: "'IP'" },
      {
        _id: "ui.device.2.parameters.8.parameter",
        value:
          "InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.1.ExternalIPAddress",
      },
      { _id: "ui.device.2.parameters.9.label", value: "'WLAN SSID'" },
      {
        _id: "ui.device.2.parameters.9.parameter",
        value: "InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.SSID",
      },
      { _id: "ui.device.2.parameters.10.label", value: "'WLAN passphrase'" },
      {
        _id: "ui.device.2.parameters.10.parameter",
        value:
          "InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.KeyPassphrase",
      },
      { _id: "ui.device.3.type", value: "'parameter-table'" },
      { _id: "ui.device.3.label", value: "'LAN hosts'" },
      {
        _id: "ui.device.3.parameter",
        value: "InternetGatewayDevice.LANDevice.1.Hosts.Host",
      },
      { _id: "ui.device.3.childParameters.0.label", value: "'Host name'" },
      { _id: "ui.device.3.childParameters.0.parameter", value: "HostName" },
      { _id: "ui.device.3.childParameters.1.label", value: "'IP address'" },
      { _id: "ui.device.3.childParameters.1.parameter", value: "IPAddress" },
      { _id: "ui.device.3.childParameters.2.label", value: "'MAC address'" },
      { _id: "ui.device.3.childParameters.2.parameter", value: "MACAddress" },
      { _id: "ui.device.4.type", value: "'container'" },
      { _id: "ui.device.4.element", value: "'div'" },
      { _id: "ui.device.4.components.0.components.0", value: "'Faults'" },
      { _id: "ui.device.4.components.0.element", value: "'h3'" },
      { _id: "ui.device.4.components.0.type", value: "'container'" },
      { _id: "ui.device.4.components.1.type", value: "'device-faults'" },
      { _id: "ui.device.5.type", value: "'container'" },
      { _id: "ui.device.5.element", value: "'div.container-full-width'" },
      {
        _id: "ui.device.5.components.0.components.0",
        value: "'All parameters'",
      },
      { _id: "ui.device.5.components.0.element", value: "'h3'" },
      { _id: "ui.device.5.components.0.type", value: "'container'" },
      { _id: "ui.device.5.components.1.type", value: "'all-parameters'" },
      { _id: "ui.device.6.type", value: "'device-actions'" },
    ]);
  }
  if (options.index) {
    resources["config"] = (resources["config"] || []).concat([
      { _id: "ui.index.0.type", value: "'device-link'" },
      { _id: "ui.index.0.label", value: "'Serial number'" },
      { _id: "ui.index.0.parameter", value: "DeviceID.SerialNumber" },
      { _id: "ui.index.0.components.0.type", value: "'parameter'" },
      { _id: "ui.index.1.label", value: "'Product class'" },
      { _id: "ui.index.1.parameter", value: "DeviceID.ProductClass" },
      { _id: "ui.index.2.label", value: "'Software version'" },
      {
        _id: "ui.index.2.parameter",
        value: "InternetGatewayDevice.DeviceInfo.SoftwareVersion",
      },
      { _id: "ui.index.3.label", value: "'IP'" },
      {
        _id: "ui.index.3.parameter",
        value:
          "InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANIPConnection.1.ExternalIPAddress",
      },
      { _id: "ui.index.4.label", value: "'SSID'" },
      {
        _id: "ui.index.4.parameter",
        value: "InternetGatewayDevice.LANDevice.1.WLANConfiguration.1.SSID",
      },
      { _id: "ui.index.5.type", value: "'container'" },
      { _id: "ui.index.5.label", value: "'Last inform'" },
      { _id: "ui.index.5.element", value: "'span.inform'" },
      { _id: "ui.index.5.parameter", value: "DATE_STRING(Events.Inform)" },
      { _id: "ui.index.5.components.0.type", value: "'parameter'" },
      { _id: "ui.index.5.components.1.chart", value: "'online'" },
      { _id: "ui.index.5.components.1.type", value: "'overview-dot'" },
      { _id: "ui.index.6.type", value: "'tags'" },
      { _id: "ui.index.6.label", value: "'Tags'" },
      { _id: "ui.index.6.parameter", value: "Tags" },
      { _id: "ui.index.6.unsortable", value: "true" },
      { _id: "ui.index.6.writable", value: "false" },
    ]);
  }
  if (options.overview) {
    resources["config"] = (resources["config"] || []).concat([
      { _id: "ui.overview.charts.online.label", value: "'Online status'" },
      {
        _id: "ui.overview.charts.online.slices.1_onlineNow.color",
        value: "'#31a354'",
      },
      {
        _id: "ui.overview.charts.online.slices.1_onlineNow.filter",
        value: "Events.Inform > NOW() - 5 * 60 * 1000",
      },
      {
        _id: "ui.overview.charts.online.slices.1_onlineNow.label",
        value: "'Online now'",
      },
      {
        _id: "ui.overview.charts.online.slices.2_past24.color",
        value: "'#a1d99b'",
      },
      {
        _id: "ui.overview.charts.online.slices.2_past24.filter",
        value:
          "Events.Inform > (NOW() - 5 * 60 * 1000) - (24 * 60 * 60 * 1000) AND Events.Inform < (NOW() - 5 * 60 * 1000)",
      },
      {
        _id: "ui.overview.charts.online.slices.2_past24.label",
        value: "'Past 24 hours'",
      },
      {
        _id: "ui.overview.charts.online.slices.3_others.color",
        value: "'#e5f5e0'",
      },
      {
        _id: "ui.overview.charts.online.slices.3_others.filter",
        value:
          "Events.Inform < (NOW() - 5 * 60 * 1000) - (24 * 60 * 60 * 1000)",
      },
      {
        _id: "ui.overview.charts.online.slices.3_others.label",
        value: "'Others'",
      },
      { _id: "ui.overview.groups.online.label", value: "''" },
      { _id: "ui.overview.groups.online.charts.0", value: "'online'" },
    ]);
  }
  if (options.presets) {
    resources["presets"] = [
      {
        _id: "bootstrap",
        weight: 0,
        channel: "bootstrap",
        events: "0 BOOTSTRAP",
        provision: "bootstrap",
      },
      { _id: "default", weight: 0, channel: "default", provision: "default" },
      { _id: "inform", weight: 0, channel: "inform", provision: "inform" },
    ];
    resources["provisions"] = [
      { _id: "bootstrap", script: BOOTSTRAP_SCRIPT },
      { _id: "default", script: DEFAULT_SCRIPT },
      { _id: "inform", script: INFORM_SCRIPT },
    ];
  }
  if (resources["permissions"]) {
    for (const p of resources["permissions"]) {
      p["_id"] = `${p["role"]}:${p["resource"]}:${p["access"]}`;
      proms.push(putPermission(p["_id"], p));
    }
  }
  if (resources["users"]) {
    for (const u of resources["users"]) {
      u["salt"] = await generateSalt(64);
      u["password"] = await hashPassword(u["password"], u["salt"]);
      u["roles"] = (u["roles"] || []).join(",");
      u["_id"] = u["username"];
      delete u["username"];
      proms.push(putUser(u["_id"], u));
    }
  }
  if (resources["provisions"]) {
    for (const p of resources["provisions"])
      proms.push(putProvision(p["_id"], p));
  }
  if (resources["presets"])
    for (const p of resources["presets"]) proms.push(putPreset(p["_id"], p));
  if (resources["config"])
    for (const c of resources["config"]) proms.push(putConfig(c["_id"], c));
  await proms;
  await Promise.all([del("ui-local-cache-hash"), del("cwmp-local-cache-hash")]);
 }
@@ -1,112 +0,0 @@
 interface Instance {
  [name: string]: string;
 }
 export default class InstanceSet {
  declare private set: Set<Instance>;
  public constructor() {
    this.set = new Set();
  }
  public add(instance: Instance): void {
    this.set.add(instance);
  }
  public delete(instance: Instance): void {
    this.set.delete(instance);
  }
  public superset(instance: Instance): Instance[] {
    const res = [];
    for (const inst of this.set) {
      let match = true;
      for (const k in instance) {
        if (inst[k] !== instance[k]) {
          match = false;
          break;
        }
      }
      if (match) res.push(inst);
    }
    res.sort((a, b) => {
      const keysA = Object.keys(a);
      const keysB = Object.keys(b);
      if (keysA.length !== keysB.length) return keysB.length - keysA.length;
      keysA.sort();
      keysB.sort();
      for (let i = 0; i < keysA.length; ++i) {
        if (keysA[i] > keysB[i]) return 1;
        else if (keysA[i] < keysB[i]) return -1;
        else if (a[keysA[i]] > b[keysB[i]]) return 1;
        else if (a[keysA[i]] < b[keysB[i]]) return -1;
      }
      return 0;
    });
    return res;
  }
  public subset(instance: Instance): Instance[] {
    const res = [];
    for (const inst of this.set) {
      let match = true;
      for (const k in inst) {
        if (inst[k] !== instance[k]) {
          match = false;
          break;
        }
      }
      if (match) res.push(inst);
    }
    res.sort((a, b) => {
      const keysA = Object.keys(a);
      const keysB = Object.keys(b);
      if (keysA.length !== keysB.length) return keysA.length - keysB.length;
      keysA.sort();
      keysB.sort();
      for (let i = 0; i < keysA.length; ++i) {
        if (keysA[i] > keysB[i]) return 1;
        else if (keysA[i] < keysB[i]) return -1;
        else if (a[keysA[i]] > b[keysB[i]]) return 1;
        else if (a[keysA[i]] < b[keysB[i]]) return -1;
      }
      return 0;
    });
    return res;
  }
  public [Symbol.iterator](): IterableIterator<Instance> {
    return this.set.values();
  }
  public forEach(callback: (instance: Instance) => void): void {
    this.set.forEach(callback);
  }
  public values(): IterableIterator<Instance> {
    return this.set.values();
  }
  public clear(): void {
    this.set.clear();
  }
  public get size(): number {
    return this.set.size;
  }
 }
@@ -1,73 +0,0 @@
 import { setTimeoutPromise } from "./util.ts";
 import { get, set } from "./cache.ts";
 import { acquireLock, releaseLock } from "./lock.ts";
 const REFRESH = 5000;
 const EVICT_TIMEOUT = 120000;
 export class LocalCache<T> {
  private nextRefresh = 1;
  private currentRevision: string = null;
  private snapshots: Map<string, T> = new Map();
  constructor(
    private cacheKey: string,
    private callback: () => Promise<[string, T]>,
  ) {}
  async getRevision(): Promise<string> {
    if (Date.now() > this.nextRefresh) await this.refresh();
    return this.currentRevision;
  }
  hasRevision(revision: string): boolean {
    return this.snapshots.has(revision);
  }
  get(revision: string): T {
    const snapshot = this.snapshots.get(revision);
    if (!snapshot) throw new Error("Cache snapshot does not exist");
    return snapshot;
  }
  async refresh(): Promise<void> {
    if (!this.nextRefresh) {
      await setTimeoutPromise(20);
      await this.refresh();
      return;
    }
    const now = Date.now();
    if (now < this.nextRefresh) return;
    this.nextRefresh = 0;
    const dbHash = await get(this.cacheKey);
    if (this.currentRevision && dbHash === this.currentRevision) {
      this.nextRefresh = now + (REFRESH - (now % REFRESH));
      return;
    }
    const lockToken = await acquireLock(this.cacheKey, 5000);
    const [hash, snapshot] = await this.callback();
    if (this.currentRevision) {
      const r = this.currentRevision;
      const s = this.snapshots.get(r);
      setTimeout(() => {
        if (this.snapshots.get(r) === s) this.snapshots.delete(r);
      }, EVICT_TIMEOUT).unref();
    }
    this.currentRevision = hash;
    this.snapshots.set(hash, snapshot);
    if (lockToken) {
      if (hash !== dbHash) await set(this.cacheKey, hash, 300);
      await releaseLock(this.cacheKey, lockToken);
    }
    this.nextRefresh = now + (REFRESH - (now % REFRESH));
  }
 }
@@ -1,50 +0,0 @@
 import { collections } from "./db/db.ts";
 const CLOCK_SKEW_TOLERANCE = 30000;
 export async function acquireLock(
  lockName: string,
  ttl: number,
  timeout = 0,
  token = Math.random().toString(36).slice(2),
 ): Promise<string> {
  try {
    const now = Date.now();
    const r = await collections.locks.findOneAndUpdate(
      { _id: lockName, value: token },
      {
        $set: {
          expire: new Date(now + ttl + CLOCK_SKEW_TOLERANCE),
        },
        $currentDate: { timestamp: true },
      },
      { upsert: true, returnDocument: "after" },
    );
    if (Math.abs(r.value.timestamp.getTime() - now) > CLOCK_SKEW_TOLERANCE)
      throw new Error("Database clock skew too great");
  } catch (err) {
    if (err.code !== 11000) throw err;
    if (!(timeout > 0)) return null;
    const w = 50 + Math.random() * 50;
    await new Promise((resolve) => setTimeout(resolve, w));
    return acquireLock(lockName, ttl, timeout - w, token);
  }
  return token;
 }
 export async function releaseLock(
  lockName: string,
  token: string,
 ): Promise<void> {
  const res = await collections.locks.deleteOne({
    _id: lockName,
    value: token,
  });
  if (res.deletedCount !== 1) throw new Error("Lock expired");
 }
 export async function getToken(lockName: string): Promise<string> {
  const res = await collections.locks.findOne({ _id: lockName });
  return res?.value;
 }
@@ -1,328 +0,0 @@
 import * as fs from "node:fs";
 import * as os from "node:os";
 import * as config from "./config.ts";
 import { getRequestOrigin } from "./forwarded.ts";
 import {
  SessionContext,
  AcsRequest,
  CpeRequest,
  CpeFault,
  InformRequest,
  Fault,
 } from "./types.ts";
 const REOPEN_EVERY = 60000;
 const LOG_FORMAT = config.get("LOG_FORMAT");
 const ACCESS_LOG_FORMAT = config.get("ACCESS_LOG_FORMAT") || LOG_FORMAT;
 const defaultMeta: { [name: string]: any } = {};
 let LOG_SYSTEMD = false;
 let ACCESS_LOG_SYSTEMD = false;
 let LOG_FILE, ACCESS_LOG_FILE;
 declare global {
  /* eslint-disable-next-line @typescript-eslint/no-namespace */
  namespace NodeJS {
    export interface WritableStream {
      fd?: number;
    }
  }
 }
 declare module "fs" {
  interface WriteStream {
    fd?: number;
  }
 }
 let logStream = fs.createWriteStream(null, { fd: process.stderr.fd });
 let logStat = fs.fstatSync(logStream.fd);
 let accessLogStream = fs.createWriteStream(null, { fd: process.stdout.fd });
 let accessLogStat = fs.fstatSync(accessLogStream.fd);
 // Reopen if original files have been moved (e.g. logrotate)
 function reopen(): void {
  let counter = 1;
  if (LOG_FILE) {
    ++counter;
    fs.stat(LOG_FILE, (err, stat) => {
      if (err && !err.message.startsWith("ENOENT:")) throw err;
      if (!(stat && stat.dev === logStat.dev && stat.ino === logStat.ino)) {
        logStream.end();
        logStream = fs.createWriteStream(null, {
          fd: fs.openSync(LOG_FILE, "a"),
        });
        logStat = fs.fstatSync(logStream.fd);
      }
      if (--counter === 0)
        setTimeout(reopen, REOPEN_EVERY - (Date.now() % REOPEN_EVERY)).unref();
    });
  }
  if (ACCESS_LOG_FILE) {
    ++counter;
    fs.stat(ACCESS_LOG_FILE, (err, stat) => {
      if (err && !err.message.startsWith("ENOENT:")) throw err;
      if (
        !(
          stat &&
          stat.dev === accessLogStat.dev &&
          stat.ino === accessLogStat.ino
        )
      ) {
        accessLogStream.end();
        accessLogStream = fs.createWriteStream(null, {
          fd: fs.openSync(ACCESS_LOG_FILE, "a"),
        });
        accessLogStat = fs.fstatSync(accessLogStream.fd);
      }
      if (--counter === 0)
        setTimeout(reopen, REOPEN_EVERY - (Date.now() % REOPEN_EVERY)).unref();
    });
  }
  if (--counter === 0)
    setTimeout(reopen, REOPEN_EVERY - (Date.now() % REOPEN_EVERY)).unref();
 }
 export function init(service: string, version: string): void {
  defaultMeta.hostname = os.hostname();
  defaultMeta.pid = process.pid;
  defaultMeta.name = `genieacs-${service}`;
  defaultMeta.version = version;
  LOG_FILE = config.get(`${service.toUpperCase()}_LOG_FILE`);
  ACCESS_LOG_FILE = config.get(`${service.toUpperCase()}_ACCESS_LOG_FILE`);
  if (LOG_FILE) {
    logStream = fs.createWriteStream(null, { fd: fs.openSync(LOG_FILE, "a") });
    logStat = fs.fstatSync(logStream.fd);
  }
  if (ACCESS_LOG_FILE) {
    accessLogStream = fs.createWriteStream(null, {
      fd: fs.openSync(ACCESS_LOG_FILE, "a"),
    });
    accessLogStat = fs.fstatSync(accessLogStream.fd);
  }
  // Determine if logs are going to journald
  const JOURNAL_STREAM = process.env["JOURNAL_STREAM"];
  if (JOURNAL_STREAM) {
    const [dev, inode] = JOURNAL_STREAM.split(":").map(parseInt);
    LOG_SYSTEMD = logStat.dev === dev && logStat.ino === inode;
    ACCESS_LOG_SYSTEMD =
      accessLogStat.dev === dev && accessLogStat.ino === inode;
  }
  if (LOG_FILE || ACCESS_LOG_FILE)
    // Can't use setInterval as we need all workers to cehck at the same time
    setTimeout(reopen, REOPEN_EVERY - (Date.now() % REOPEN_EVERY)).unref();
 }
 export function close(): void {
  accessLogStream.end();
  logStream.end();
 }
 export function flatten(
  details: Record<string, unknown>,
 ): Record<string, unknown> {
  if (details.sessionContext) {
    const sessionContext = details.sessionContext as SessionContext;
    details.deviceId = sessionContext.deviceId;
    details.remoteAddress = getRequestOrigin(
      sessionContext.httpRequest,
    ).remoteAddress;
    delete details.sessionContext;
  }
  if (details.exception) {
    const err = details.exception as Error;
    details.exceptionName = err.name;
    details.exceptionMessage = err.message;
    details.exceptionStack = err.stack;
    delete details.exception;
  }
  if (details.task) {
    details.taskId = details.task["_id"];
    delete details.task;
  }
  if (details.rpc) {
    const rpc = details.rpc as {
      id: string;
      acsRequest?: AcsRequest;
      cpeRequest?: CpeRequest;
      cpeFault?: CpeFault;
    };
    if (rpc.acsRequest) {
      details.acsRequestId = rpc.id;
      details.acsRequestName = rpc.acsRequest.name;
      if (rpc.acsRequest["commandKey"])
        details.acsRequestCommandKey = rpc.acsRequest["commandKey"];
    } else if (rpc.cpeRequest) {
      details.cpeRequestId = rpc.id;
      if (rpc.cpeRequest.name === "Inform") {
        details.informEvent = (rpc.cpeRequest as InformRequest).event.join(",");
        details.informRetryCount = (rpc.cpeRequest as InformRequest).retryCount;
      } else {
        details.cpeRequestName = rpc.cpeRequest.name;
        if (rpc.cpeRequest["commandKey"])
          details.cpeRequestCommandKey = rpc.cpeRequest["commandKey"];
      }
    } else if (rpc.cpeFault) {
      details.acsRequestId = rpc.id;
      details.cpeFaultCode = rpc.cpeFault.detail.faultCode;
      details.cpeFaultString = rpc.cpeFault.detail.faultString;
    }
    delete details.rpc;
  }
  if (details.fault) {
    const fault = details.fault as Fault;
    details.faultCode = fault.code;
    details.faultMessage = fault.message;
    delete details.fault;
  }
  // For genieacs-ui
  if (details.context) {
    details.remoteAddress = getRequestOrigin(
      details.context["req"],
    ).remoteAddress;
    if (details.context["state"].user)
      details.user = details.context["state"].user.username;
    delete details.context;
  }
  for (const [k, v] of Object.entries(details))
    if (v == null) delete details[k];
  return details;
 }
 function formatJson(
  details: Record<string, unknown>,
  systemd: boolean,
 ): string {
  if (systemd) {
    let severity = "";
    if (details.severity === "info") severity = "<6>";
    else if (details.severity === "warn") severity = "<4>";
    else if (details.severity === "error") severity = "<3>";
    return `${severity}${JSON.stringify(flatten(details))}${os.EOL}`;
  }
  return `${JSON.stringify(flatten(details))}${os.EOL}`;
 }
 function formatSimple(
  details: Record<string, unknown>,
  systemd: boolean,
 ): string {
  const skip = {
    user: true,
    remoteAddress: true,
    severity: true,
    timestamp: true,
    message: true,
    deviceId: !!details.sessionContext,
  };
  flatten(details);
  let remote = "";
  if (details.remoteAddress) {
    if (details.deviceId && skip["deviceId"])
      remote = `${details.remoteAddress} ${details.deviceId}: `;
    else if (details.user)
      remote = `${details.user}@${details.remoteAddress}: `;
    else remote = `${details.remoteAddress}: `;
  }
  const keys = Object.keys(details);
  let meta = "";
  const kv = [];
  for (const k of keys)
    if (!skip[k]) kv.push(`${k}=${JSON.stringify(details[k])}`);
  if (kv.length) meta = `; ${kv.join(" ")}`;
  if (systemd) {
    let severity = "";
    if (details.severity === "info") severity = "<6>";
    else if (details.severity === "warn") severity = "<4>";
    else if (details.severity === "error") severity = "<3>";
    return `${severity}${remote}${details.message}${meta}${os.EOL}`;
  }
  return `${details.timestamp} [${(
    details.severity as string
  ).toUpperCase()}] ${remote}${details.message}${meta}${os.EOL}`;
 }
 function log(details: Record<string, unknown>): void {
  details.timestamp = new Date().toISOString();
  if (LOG_FORMAT === "json") {
    details = Object.assign({}, defaultMeta, details);
    logStream.write(formatJson(details, LOG_SYSTEMD));
  } else {
    logStream.write(formatSimple(details, LOG_SYSTEMD));
  }
 }
 export function info(details: Record<string, unknown>): void {
  details.severity = "info";
  log(details);
 }
 export function warn(details: Record<string, unknown>): void {
  details.severity = "warn";
  log(details);
 }
 export function error(details: Record<string, unknown>): void {
  details.severity = "error";
  log(details);
 }
 export function accessLog(details: Record<string, unknown>): void {
  details.timestamp = new Date().toISOString();
  if (ACCESS_LOG_FORMAT === "json") {
    Object.assign(details, defaultMeta);
    accessLogStream.write(formatJson(details, ACCESS_LOG_SYSTEMD));
  } else {
    accessLogStream.write(formatSimple(details, ACCESS_LOG_SYSTEMD));
  }
 }
 export function accessInfo(details: Record<string, unknown>): void {
  details.severity = "info";
  accessLog(details);
 }
 export function accessWarn(details: Record<string, unknown>): void {
  details.severity = "warn";
  accessLog(details);
 }
 export function accessError(details: Record<string, unknown>): void {
  details.severity = "error";
  accessLog(details);
 }
@@ -0,0 +1,141 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This file incorporates work covered by the following copyright and
 # permission notice:
 #
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 common = require './common'
 test = (obj, query) ->
  for k,v of query
    if k.charAt(0) == '$' # this is a logical operator
      res = testExpressions(obj, v, k)
    else
      value = common.getParamValueFromPath(obj, k)
      if common.typeOf(v) isnt common.OBJECT_TYPE
        # TODO comparing array to regex, array to array, and object to object
        if common.typeOf(value) is common.ARRAY_TYPE
          res = value.indexOf(v) != -1
        else
          if common.typeOf(v) is common.REGEXP_TYPE
            res = v.test(value)
          else
            res = v == value
      else
        for k2,v2 of v
          switch k2
            when '$ne'
              if common.typeOf(value) is common.ARRAY_TYPE
                res = value.indexOf(v2) == -1
              else
                res = value != v2
            when '$lt'
              res = value < v2
            when '$lte'
              res = value <= v2
            when '$gt'
              res = value > v2
            when '$gte'
              res = value >= v2
            when '$regex'
              res = v2.test(value)
            when '$in'
              throw new Error('Operator not supported')        
            when '$nin'
              throw new Error('Operator not supported')        
            when '$all'
              throw new Error('Operator not supported')        
            when '$exists'
              throw new Error('Operator not supported')        
            else
              throw new Error('Operator not supported')        
    if not res
      return false
  return true
 testExpressions = (obj, expressions, lop) ->
  for f in expressions
    res = test(obj, f)
    switch lop
      when '$and'
        return false if not res
      when '$or'
        return true if res
      when '$nor'
        return false if res
  return switch lop
    when '$and' then true
    when '$or' then false
    when '$nor' then true
    else throw new Error('Unknown logical operator')
 # generate field projection from a query
 projection = (query, proj) ->
  for k,v of query
    if k.charAt(0) == '$' # this is a logical operator
      for q in v
        projection(q, proj)
    else
      proj[k] = 1
 # optimize projection by removing overlaps
 optimizeProjection = (obj) ->
  keys = Object.keys(obj).sort()
  return if keys.length <= 1
  i = 1
  while i < keys.length
    a = keys[i-1]
    b = keys[i]
    if common.startsWith(b, a)
      if b.charAt(a.length) == '.' or b.charAt(a.length - 1) == '.'
        delete obj[b]
        keys.splice(i--, 1)
    ++ i
  return
 exports.test = test
 exports.projection = projection
 exports.optimizeProjection = optimizeProjection
@@ -0,0 +1,459 @@
 ###
 # Copyright 2013, 2014  Zaid Abdulla
 #
 # GenieACS is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
 # published by the Free Software Foundation, either version 3 of the
 # License, or (at your option) any later version.
 #
 # GenieACS is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU Affero General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with GenieACS.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This file incorporates work covered by the following copyright and
 # permission notice:
 #
 # Copyright 2013 Fanoos Telecom
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
 # in the Software without restriction, including without limitation the rights
 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 # copies of the Software, and to permit persons to whom the Software is
 # furnished to do so, subject to the following conditions:
 #
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 #
 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 ###
 config = require './config'
 url = require 'url'
 common = require './common'
 db = require './db'
 mongodb = require 'mongodb'
 querystring = require 'querystring'
 query = require './query'
 apiFunctions = require './api-functions'
 presets = require './presets'
 # regular expression objects
 DEVICE_TASKS_REGEX = /^\/devices\/([a-zA-Z0-9\-\_\%]+)\/tasks\/?$/
 TASKS_REGEX = /^\/tasks\/([a-zA-Z0-9\-\_\%]+)(\/[a-zA-Z_]*)?$/
 TAGS_REGEX = /^\/devices\/([a-zA-Z0-9\-\_\%]+)\/tags\/([a-zA-Z0-9\-\_\%]+)\/?$/
 PRESETS_REGEX = /^\/presets\/([a-zA-Z0-9\-\_\%]+)\/?$/
 OBJECTS_REGEX = /^\/objects\/([a-zA-Z0-9\-\_\%]+)\/?$/
 FILES_REGEX = /^\/files\/([a-zA-Z0-9\%\!\*\'\(\)\;\:\@\&\=\+\$\,\?\#\[\]\-\_\.\~]+)\/?$/
 PING_REGEX = /^\/ping\/([a-zA-Z0-9\-\_\.]+)\/?$/
 QUERY_REGEX = /^\/([a-zA-Z0-9_]+s)\/?$/
 DEVICE_PRESET_REGEX = /^\/devices\/([a-zA-Z0-9\-\_\%]+)\/preset\/?$/
 DELETE_DEVICE_REGEX = /^\/devices\/([a-zA-Z0-9\-\_\%]+)\/?$/
 errorToString = (err) ->
  "#{err.name}: #{err.message}"
 listener = (request, response) ->
  chunks = []
  bytes = 0
  request.addListener 'data', (chunk) ->
    chunks.push(chunk)
    bytes += chunk.length
  request.getBody = () ->
    # Write all chunks into a Buffer
    body = new Buffer(bytes)
    offset = 0
    chunks.forEach((chunk) ->
      chunk.copy(body, offset, 0, chunk.length)
      offset += chunk.length
    )
    return body
  request.addListener 'end', () ->
    body = request.getBody()
    urlParts = url.parse(request.url, true)
    if PRESETS_REGEX.test(urlParts.pathname)
      presetName = querystring.unescape(PRESETS_REGEX.exec(urlParts.pathname)[1])
      if request.method == 'PUT'
        preset = JSON.parse(body)
        preset._id = presetName
        db.presetsCollection.save(preset, (err) ->
          db.redisClient.del('presets', 'presets_hash', (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          response.writeHead(200)
          response.end()
        )
      else if request.method == 'DELETE'
        db.presetsCollection.remove({'_id' : presetName}, (err, removedCount) ->
          db.redisClient.del('presets', 'presets_hash', (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          if removedCount == 0
            response.writeHead(404)
            response.end()
            return
          response.writeHead(200)
          response.end()
        )
      else
        response.writeHead 405, {'Allow': 'PUT, DELETE'}
        response.end('405 Method Not Allowed')
    else if OBJECTS_REGEX.test(urlParts.pathname)
      objectName = querystring.unescape(OBJECTS_REGEX.exec(urlParts.pathname)[1])
      if request.method == 'PUT'
        object = JSON.parse(body)
        object._id = objectName
        db.objectsCollection.save(object, (err) ->
          db.redisClient.del('objects', 'presets_hash', (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          response.writeHead(200)
          response.end()
        )
      else if request.method == 'DELETE'
        db.objectsCollection.remove({'_id' : objectName}, (err, removedCount) ->
          db.redisClient.del('objects', 'presets_hash', (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          if removedCount == 0
            response.writeHead(404)
            response.end()
            return
          response.writeHead(200)
          response.end()
        )
      else
        response.writeHead 405, {'Allow': 'PUT, DELETE'}
        response.end('405 Method Not Allowed')
    else if TAGS_REGEX.test(urlParts.pathname)
      r = TAGS_REGEX.exec(urlParts.pathname)
      deviceId = querystring.unescape(r[1])
      tag = querystring.unescape(r[2])
      if request.method == 'POST'
        db.devicesCollection.update({'_id' : deviceId}, {'$addToSet' : {'_tags' : tag}}, {safe: true}, (err, updatedCount) ->
          db.redisClient.del("#{deviceId}_presets_hash", (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          if updatedCount == 0
            response.writeHead(404)
            response.end()
            return
          response.writeHead(200)
          response.end()
        )
      else if request.method == 'DELETE'
        db.devicesCollection.update({'_id' : deviceId}, {'$pull' : {'_tags' : tag}}, {safe: true}, (err, updatedCount) ->
          db.redisClient.del("#{deviceId}_presets_hash", (err) ->
            throw err if err
          )
          if err
            response.writeHead(500)
            response.end(errorToString(err))
            return
          if updatedCount == 0
            response.writeHead(404)
            response.end()
            return
          response.writeHead(200)
          response.end()
        )
      else
        response.writeHead 405, {'Allow': 'POST, DELETE'}
        response.end('405 Method Not Allowed')
    else if DEVICE_TASKS_REGEX.test(urlParts.pathname)
      if request.method == 'POST'
        deviceId = querystring.unescape(DEVICE_TASKS_REGEX.exec(urlParts.pathname)[1])
        if body.length
          task = JSON.parse(body)
          task.device = deviceId
          db.getAliases((aliases) ->
            apiFunctions.insertTasks(task, aliases, (err) ->
              db.redisClient.del("#{deviceId}_presets_hash", (err) ->
                throw err if err
              )
              if err
                response.writeHead(500)
                response.end(errorToString(err))
                return
              if urlParts.query.connection_request?
                apiFunctions.connectionRequest(deviceId, (err) ->
                  if err
                    response.writeHead(202, err.message, {'Content-Type' : 'application/json'})
                    response.end(JSON.stringify(task))
                  else
                    apiFunctions.watchTask(task._id, config.get('DEVICE_ONLINE_THRESHOLD', deviceId), (err, status) ->
                      if err
                        response.writeHead(500)
                        response.end(errorToString(err))
                        return
                      if status is 'timeout'
                        response.writeHead(202, 'Task queued but not processed', {'Content-Type' : 'application/json'})
                        response.end(JSON.stringify(task))
                      else if status is 'fault'
                        db.tasksCollection.findOne({_id : task._id}, (err, task) ->
                          if err
                            response.writeHead(500)
                            response.end(errorToString(err))
                            return
                          response.writeHead(202, 'Task faulted', {'Content-Type' : 'application/json'})
                          response.end(JSON.stringify(task))
                        )
                      else
                        response.writeHead(200, {'Content-Type' : 'application/json'})
                        response.end(JSON.stringify(task))
                    )
                )
              else
                response.writeHead(202, {'Content-Type' : 'application/json'})
                response.end(JSON.stringify(task))
            )
          )
        else if urlParts.query.connection_request?
          # no task, send connection request only
          apiFunctions.connectionRequest(deviceId, (err) ->
            if err
              response.writeHead 504
              response.end(errorToString(err))
              return
            response.writeHead 200
            response.end()
          )
        else
          response.writeHead(400)
          response.end()
      else
        response.writeHead 405, {'Allow': 'POST'}
        response.end('405 Method Not Allowed')
    else if DEVICE_PRESET_REGEX.test(urlParts.pathname)
      deviceId = querystring.unescape(DEVICE_PRESET_REGEX.exec(urlParts.pathname)[1])
      if request.method is 'GET'
        db.getPresetsObjectsAliases((allPresets, allObjects, allAliases) ->
          presets.getDevicePreset(deviceId, allPresets, allObjects, allAliases, (devicePreset) ->
            response.writeHead(200, {'Content-Type' : 'application/json'})
            response.end(JSON.stringify(devicePreset))
          )
        )
      else
        response.writeHead 405, {'Allow': 'GET'}
        response.end('405 Method Not Allowed')
    else if TASKS_REGEX.test(urlParts.pathname)
      r = TASKS_REGEX.exec(urlParts.pathname)
      taskId = mongodb.ObjectID(querystring.unescape(r[1]))
      action = r[2]
      if not action? or action is '/'
        if request.method == 'DELETE'
          db.tasksCollection.remove({'_id' : taskId}, (err, removedCount) ->
            if err
              response.writeHead(500)
              response.end(errorToString(err))
              return
            if removedCount == 0
              response.writeHead(404)
              response.end()
              return
            response.writeHead(200)
            response.end()
          )
        else
          response.writeHead 405, {'Allow': 'PUT DELETE'}
          response.end('405 Method Not Allowed')
      else if action is '/retry'
        if request.method == 'POST'
          db.tasksCollection.update({_id : taskId}, {$unset : {fault : 1}, $set : {timestamp : new Date()}}, (err, updatedCount) ->
            # TODO need to invalidate presets hash for the device
            throw err if err
            if updatedCount == 0
              response.writeHead(404)
              response.end()
              return
            response.writeHead(200)
            response.end()
          )
        else
          response.writeHead 405, {'Allow': 'POST'}
          response.end('405 Method Not Allowed')
      else
        response.writeHead(404)
        response.end()
    else if FILES_REGEX.test(urlParts.pathname)
      filename = querystring.unescape(FILES_REGEX.exec(urlParts.pathname)[1])
      if request.method == 'PUT'
        metadata = {
          fileType : request.headers.filetype,
          oui : request.headers.oui,
          productClass : request.headers.productclass,
          version : request.headers.version,
        }
        gs = new mongodb.GridStore(db.mongoDb, filename, 'w', {metadata : metadata})
        gs.open((err, gs) ->
          gs.write(body, (err, res) ->
            throw err if err
            gs.close((err) ->
              throw err if err
              response.writeHead(201)
              response.end()
            )
          )
        )
      else if request.method == 'DELETE'
        mongodb.GridStore.unlink(db.mongoDb, filename, (err) ->
          response.writeHead(200)
          response.end()
        )
      else
        response.writeHead 405, {'Allow': 'PUT, DELETE'}
        response.end('405 Method Not Allowed')
    else if PING_REGEX.test(urlParts.pathname)
      host = querystring.unescape(PING_REGEX.exec(urlParts.pathname)[1])
      require('child_process').exec("ping -w 1 -i 0.2 -c 3 #{host}", (err, stdout, stderr) ->
        if err
          response.writeHead(404, {'Cache-Control' : 'no-cache'})
          response.end(errorToString(err))
          return
        response.writeHead(200, {'Content-Type' : 'text/plain', 'Cache-Control' : 'no-cache'})
        response.end(stdout)
      )
    else if DELETE_DEVICE_REGEX.test(urlParts.pathname)
      if request.method isnt 'DELETE'
        response.writeHead(405, {'Allow' : 'DELETE'})
        response.end('405 Method Not Allowed')
        return
      deviceId = querystring.unescape(DELETE_DEVICE_REGEX.exec(urlParts.pathname)[1])
      apiFunctions.deleteDevice(deviceId, (err) ->
        if err
          response.writeHead(500)
          response.end(errorToString(err))
          return
        response.writeHead(200)
        response.end()
      )
    else if QUERY_REGEX.test(urlParts.pathname)
      collectionName = QUERY_REGEX.exec(urlParts.pathname)[1]
      if request.method not in ['GET', 'HEAD']
        response.writeHead 405, {'Allow' : 'GET, HEAD'}
        response.end('405 Method Not Allowed')
        return
      collection = db["#{collectionName}Collection"]
      if not collection?
        response.writeHead 404
        response.end('404 Not Found')
        return
      func = (aliases) ->
        if urlParts.query.query?
          try
            q = JSON.parse(urlParts.query.query)
          catch err
            response.writeHead(400)
            response.end(errorToString(err))
            return
        else
          q = {}
        q = query.expand(q, aliases) if collectionName is 'devices'
        q = query.substituteObjectId(q) if collectionName is 'tasks'
        if urlParts.query.projection?
          projection = {}
          for p in urlParts.query.projection.split(',')
            p = p.trim()
            projection[p] = 1
            if collectionName is 'devices'
              for k,v of aliases
                if k == p or common.startsWith(k, "#{p}.")
                  projection[a] = 1 for a in v
        cur = collection.find(q, projection, {batchSize : 50})
        if urlParts.query.sort?
          s = JSON.parse(urlParts.query.sort)
          sort = {}
          for k,v of s
            if aliases[k]?
              sort[a] = v for a in aliases[k]
            else
              sort[k] = v
          cur.sort(sort)
        cur.skip(parseInt(urlParts.query.skip)) if urlParts.query.skip?
        cur.limit(parseInt(urlParts.query.limit)) if urlParts.query.limit?
        cur.count((err, total) ->
          response.writeHead(200, {'Content-Type' : 'application/json', 'total' : total})
          if request.method is 'HEAD'
            response.end()
            return
          response.write("[\n")
          i = 0
          cur.each((err, item) ->
            if item is null
              response.end("\n]")
            else
              response.write(",\n") if i++
              apiFunctions.addAliases(item, aliases) if collectionName is 'devices'
              response.write(JSON.stringify(item))
          )
        )
      if collectionName is 'devices'
        db.getAliases((aliases) ->
          func(aliases)
        )
      else
        func({})
    else
      response.writeHead 404
      response.end('404 Not Found')
 exports.listener = listener
Author	SHA1	Message	Date
Zaid Abdulla	a47811d20f	1.0.5	2015-11-22 01:46:35 +03:00
Zaid Abdulla	86833a7011	Use a valid SPDX license expression	2015-11-07 20:35:10 +03:00
Zaid Abdulla	7032f273db	Merge pull request #110 from shweelan/v1.0 Respond with 404 when manipulating resources that don't exist.	2015-09-26 21:57:19 +03:00
Shweelan L. Samson	6ae4728ded	response 404 when device not found or task not found or object not found, when update or delete	2015-09-07 14:11:20 +03:00
Zaid Abdulla	cda98e3145	Return task object including fault details in case of fault. Return reason for connection request failure.	2015-07-09 22:22:04 +03:00
Zaid Abdulla	b2609c228a	Fixed error details not being properly saved in fault property of customCommand tasks.	2015-07-05 03:44:44 +03:00
Zaid Abdulla	52fb326942	Ensure customCommand init tasks triggered by inform run after any refreshObject task.	2015-07-05 01:39:54 +03:00
Zaid Abdulla	28c5dc0d46	Fixed bug where ordering of key parameters in object definition may affect preset behavior.	2015-07-03 01:42:49 +03:00
Zaid Abdulla	a02e3ecf17	Fixed the incorrect way used to return error resonses in NBI.	2015-07-01 05:11:30 +03:00
Zaid Abdulla	b91bcb7442	1.0.4	2015-05-31 19:40:33 +03:00
Zaid Abdulla	a0c0a7938b	Updated dependency version ranges.	2015-05-31 19:39:12 +03:00
Dan Morphis	82f354bf2b	Set the status code to 400 when invalid query params are sent	2015-05-31 13:38:47 +03:00
Dan Morphis	7f64b567db	Don't crash if the query params are invalid json	2015-05-31 13:38:47 +03:00
Dan Morphis	b17f842de7	Return the task object when adding a task	2015-05-31 13:38:15 +03:00
Zaid Abdulla	ff68553bd7	No longer storing or relying on object instance names as that proved to be unnecessary and can potentially cause unpredictable behavior in certain edge cases.	2015-05-31 03:04:26 +03:00
Zaid Abdulla	22b11d6103	Updated README file.	2015-05-31 00:24:05 +03:00
Zaid Abdulla	00987ea8e2	Searching by MAC now works regardless of whether the device's MAC is lower or upper case.	2015-05-30 01:49:52 +03:00
Zaid Abdulla	f0a1289818	Modified sample configuration for wider device compatibility.	2015-05-29 14:21:16 +03:00
Zaid Abdulla	38840d5b8d	Added debug dump files to .gitignore.	2015-05-29 14:12:19 +03:00
Zaid Abdulla	74008bc7de	Implemented a workaround for certain devices dropping connection after digest authentication challege in connection requests.	2015-05-29 14:11:05 +03:00
Zaid Abdulla	305558f186	Fixed ability to query tasks collection using task ID. Fixes #68 .	2015-05-26 12:54:04 +03:00
Zaid Abdulla	4ad8aa5a1b	Fixed bug in addObject task where a given parameter value may be set to all descendant parameters wth the same name.	2015-05-26 00:44:50 +03:00
Zaid Abdulla	c70354997a	Support nested parameters in object configurations in presets.	2015-05-25 11:22:53 +03:00
Zaid Abdulla	c88e5d6fe8	Fixed bug causing only a single preset configuration of custom_command_age and custom_comand_value to take affect.	2015-02-01 19:59:53 +03:00
Zaid Abdulla	4e9186812c	1.0.3	2015-01-13 23:01:21 +03:00
Zaid Abdulla	41740cbf0b	Fixed 404 error when deleting/retrying a task caused by recent change in node's implementation of querystring.unescape.	2014-11-22 19:11:41 +03:00
Zaid Abdulla	2932b9ab6d	Correctly use https scheme for download RPC when FS_SSL config is set. Fixes #43 .	2014-11-14 17:23:08 +03:00
Zaid Abdulla	3625f26134	1.0.2	2014-09-07 20:46:13 +03:00
Zaid Abdulla	c1d486b776	Updated readme file.	2014-09-07 20:45:17 +03:00
Zaid Abdulla	ae441beda0	1.0.1	2014-09-05 16:06:45 +03:00
		`@@ -1 +0,0 @@`
			<svg xmlns="http://www.w3.org/2000/svg" width="310" height="64" viewBox="0 0 82.021 16.933"><path d="M7.341 9.892l.687 1.307c-.305.53-.718.99-1.238 1.378-.851.503-1.543.421-2.076-.244 1.03-.466 1.907-1.28 2.627-2.44M4.676 5.97a2.144 2.144 0 0 1 .237 1.228 8.358 8.358 0 0 1-.518 2.21 7.386 7.386 0 0 0-.424 1.708 2.38 2.38 0 0 0 .168 1.205c.163.393.457.73.883 1.01-.458.138-.859.175-1.202.112-.6-.114-.992-.476-1.175-1.084a3.571 3.571 0 0 1-.093-1.787 11.742 11.742 0 0 1 .435-1.644 11.606 11.606 0 0 0 .432-1.708c.037-.22.041-.45.01-.687a1.66 1.66 0 0 0-.273-.706 1.582 1.582 0 0 0-.6-.506c.245-.046.487-.083.728-.113a1.45 1.45 0 0 1 .728.123c.295.13.517.343.665.639m7.901-1.802l.222.218-1.847 1.678.555.555 1.678-1.846.218.22-1.843 1.679.521.521L13.76 5.35l.222.218-1.847 1.678.556.556 1.764-1.765.375.368-1.584 1.584h.004l-2.065 2.061-1.253.515-1.795-1.348-.007-.008-.008.008.004.007 1.194 1.592-.398.161-1.75-1.75.444-1.08 1.636 1.228.004.003.011-.007-.003-.004-1.39-1.85.225-.553 1.694-1.689h-.004l1.956-1.952.372.368-1.769 1.768.552.552 1.678-1.843m1.456 9.91c1.55-1.549 2.324-3.42 2.324-5.612 0-2.187-.774-4.058-2.323-5.612C12.48 1.304 10.61.53 8.423.53 6.23.53 4.36 1.305 2.81 2.854c-.95.953-1.609 2.025-1.974 3.213l.158-.086c.788-.415 1.486-.203 2.094.638-.55.113-.992.232-1.325.357a1.923 1.923 0 0 0-.503.289c-.337.268-.55.608-.638 1.02a3.98 3.98 0 0 0 .011 1.761c.285 1.527 1.011 2.87 2.178 4.032 1.549 1.552 3.42 2.328 5.612 2.328 2.19 0 4.06-.776 5.612-2.328" fill="white"/><path d="M47.104 1.767c-.456 0-.816.388-.816.849 0 .46.355.816.816.816a.82.82 0 0 0 .833-.816.85.85 0 0 0-.833-.849zM25.602 3.171c-.205 0-.382.046-.5.13-.118.085-.187.201-.187.4v.981l-.151-.031a9.89 9.89 0 0 0-1.872-.173c-1.288 0-2.148.066-2.657.374a1.264 1.264 0 0 0-.548.676c-.119.313-.177.73-.177 1.28v6.123c0 .982.25 1.508.787 1.824.539.315 1.407.41 2.628.41 1.308 0 2.178-.13 2.701-.484.524-.354.745-.932.745-1.942 0-.821-.147-1.37-.543-1.722-.395-.352-1.075-.528-2.186-.528h-2.787V9.255l.148.03c.478.092 1.094.11 1.89.11 1.292 0 2.148-.112 2.658-.453.51-.34.724-.915.724-1.958V3.7c0-.198-.07-.314-.186-.399a.843.843 0 0 0-.487-.13zm6.112 1.29c-1.226 0-2.038.137-2.526.495-.488.357-.697.945-.697 1.947v2.84c0 .935.23 1.54.743 1.932.512.39 1.337.571 2.544.571 1.392 0 2.17-.058 2.567-.196.199-.069.298-.15.355-.246.058-.097.078-.227.078-.405 0-.208-.04-.378-.117-.485-.076-.108-.186-.171-.396-.171-.794 0-1.057.19-2.408.19-.593 0-1.06-.078-1.395-.255-.335-.177-.53-.473-.53-.84v-.89h4.301a.717.717 0 0 0 .539-.213c.13-.138.213-.342.213-.62V6.903c0-1.037-.179-1.623-.654-1.97-.476-.348-1.3-.472-2.617-.472zm8.83 0c-1.303 0-2.117.086-2.59.49-.472.403-.665 1.168-.665 2.623v4.002c0 .215.074.338.197.426.123.088.31.134.524.134.223 0 .412-.046.537-.134.125-.088.198-.211.198-.426v-3.81c0-.424.006-.754.04-1.019.036-.265.102-.47.236-.623.135-.152.328-.236.574-.28.246-.044.552-.053.949-.053.4 0 .712.01.963.053.25.044.447.126.587.278.139.151.211.358.25.623.038.265.046.597.046 1.02v3.81c0 .217.07.34.19.427.122.088.308.134.53.134.215 0 .401-.046.525-.134.123-.088.196-.211.196-.426V7.574c0-1.455-.201-2.219-.682-2.623-.48-.404-1.303-.49-2.605-.49zm12.996 0c-1.227 0-2.038.137-2.527.495-.488.357-.696.945-.696 1.947v2.84c0 .935.23 1.54.742 1.932.513.39 1.338.571 2.545.571 1.392 0 2.17-.058 2.567-.196.2-.069.298-.15.355-.246.058-.097.079-.227.079-.405 0-.208-.04-.378-.117-.485-.077-.108-.188-.171-.398-.171-.793 0-1.056.19-2.406.19-.594 0-1.062-.078-1.396-.255-.335-.177-.53-.473-.53-.84v-.89h4.301a.716.716 0 0 0 .539-.213c.13-.138.213-.342.213-.62V6.903c0-1.037-.18-1.623-.655-1.97-.475-.348-1.3-.472-2.616-.472zm-6.436.13c-.212 0-.398.047-.522.14-.124.09-.198.22-.198.435v6.41c0 .215.073.338.196.426.123.088.31.134.524.134.223 0 .414-.046.54-.134.124-.088.198-.211.198-.426v-6.41c0-.215-.075-.342-.201-.434-.126-.092-.316-.142-.537-.142zM22.892 5.726c.662 0 1.4.065 1.918.146l.105.016v.92c0 .246-.01.453-.058.632a.757.757 0 0 1-.292.441c-.305.216-.798.263-1.673.263h-.123v-.013c-.761 0-1.26-.002-1.55-.16a.637.637 0 0 1-.307-.395c-.049-.17-.057-.37-.057-.625v-.062c0-.227.03-.418.108-.58a.811.811 0 0 1 .373-.373c.337-.171.826-.21 1.556-.21zm8.839.049c.677 0 1.116.054 1.412.237a.775.775 0 0 1 .316.373c.063.154.085.33.085.535v.811h-3.611V6.92c0-.437.135-.755.439-.928.304-.174.739-.217 1.359-.217zm21.825 0c.677 0 1.117.054 1.413.237a.779.779 0 0 1 .318.373c.062.154.084.33.084.535v.811h-3.613V6.92c0-.437.136-.755.44-.928.304-.174.738-.217 1.358-.217zm-32.701 5.867h2.388c.613 0 1.05.045 1.355.226.306.18.443.505.443.95 0 .429-.179.743-.533.907-.354.164-.861.206-1.568.206-.652 0-1.151-.045-1.51-.186a.952.952 0 0 1-.425-.3.789.789 0 0 1-.15-.483v-1.195z" fill="white"/><path d="M62.339 4.462c-1.399 0-2.18.078-2.58.225-.2.073-.3.157-.357.254-.058.097-.079.22-.079.386 0 .199.043.362.122.469.079.107.19.172.392.172.38 0 .616-.048.947-.096a9.457 9.457 0 0 1 1.475-.098c.58 0 .966.043 1.233.192a.685.685 0 0 1 .292.312c.06.13.082.281.082.452v.857h-2.484c-1.134 0-1.763.19-2.11.559-.346.367-.443.95-.443 1.802 0Line truncated