Jesse A. Green / notebook

Live demo: the DocCompiler guide library as a static, read-only snapshot (October 2026). No backend, no accounts, nothing to submit.

Network Sketcher Administrator & Developer Guide

Generated April 2026 from cisco-open/network-sketcher at ae4fafb

AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it.

This guide is the operational and architectural reference for engineers who run Network Sketcher in shared environments, integrate it with surrounding systems, or extend it. It complements the User Guide: where the User Guide answers "how do I draw a diagram?", this guide answers "how does the engine work, what does every config knob do, what are the failure modes, and where do I patch behavior?".

Note

The repository ships two editions that share a common rendering engine. Operational concerns differ — the Online edition is what you deploy on a server; the Offline edition is what users run on their workstations. This guide is structured for the Online edition and calls out Offline differences explicitly where they matter.


1. Architecture Overview

Network Sketcher Online is a single-process Flask web service that fronts a deterministic Python rendering engine. The engine is the same code in both editions; the Online edition wraps it in headless stubs (nsm_adapter.py) so it can run without a display server.

1.1 What it does

A user uploads a [MASTER]*.xlsx (or .nsm) file. The engine reads its three well-known sheets (Master_Data, Master_Data_L2, Master_Data_L3), decomposes them into in-memory section arrays, and runs one of three dedicated diagram pipelines (L1, L2, L3) against them. Output is written as PowerPoint (.pptx), SVG, or Excel ([DEVICE]*.xlsx) and offered for download. A CLI layer (add/delete/rename/show/export) lets users mutate the master file by issuing line-oriented commands; mutations are versioned so the original is preserved.

1.2 ASCII architecture

                   ┌─────────────────────────────────────────────────┐
                   │              Browser (HTTPS, :5443)             │
                   │  Single-page UI · SSE · localStorage(job_id)    │
                   └────────────────────┬────────────────────────────┘
                                        │ TLS (self-signed by default)
                   ┌────────────────────▼────────────────────────────┐
                   │           ns_web_start.py (Flask app)           │
                   │ Routes: /upload /generate_step /run_commands    │
                   │   /svg_grid_stream /download(_visio|_drawio|all)│
                   │ Headers: HSTS · X-Frame-Options · CSP           │
                   │ Per-job state: uploads/<uuid>/                  │
                   │ Heartbeats: in-memory dict, 5-min idle expiry   │
                   └────┬───────────────┬───────────────┬────────────┘
                        │               │               │
        in-process run_cli              │   subprocess(_ns_worker.py)
                        │               │               │
                   ┌────▼───────────────▼───────────────▼────────────┐
                   │          ns_engine/  (headless engine)          │
                   │ nsm_adapter.py    Tk stubs + bootstrap          │
                   │ nsm_cli.py        verb dispatcher (5 verbs)     │
                   │ nsm_io.py         .xlsx ↔ .nsm (Parquet) bridge │
                   │ nsm_l1_*.py       L1 diagram + table + sync     │
                   │ nsm_l2_*.py       L2 diagram + table + sync     │
                   │ nsm_l3_*.py       L3 diagram + table + sync     │
                   │ nsm_ddx_*.py      pptx / svg renderers          │
                   │ nsm_extensions.py AI-context · flow · IP report │
                   │ nsm_sync_between_layers.py  L2↔L3 consistency   │
                   └────┬───────────────────────┬─────────────────────┘
                        │                       │
                ┌───────▼─────────┐    ┌────────▼───────────────────┐
                │ Master file     │    │ Output artifacts           │
                │ [MASTER]*.xlsx  │    │ [L1_DIAGRAM]*.pptx/.svg    │
                │  or *.nsm (zip  │    │ [L2_DIAGRAM]<area>_*       │
                │  of Parquet)    │    │ [L3_DIAGRAM]*              │
                │ ─ Master_Data   │    │ [DEVICE]*.xlsx             │
                │ ─ Master_Data_L2│    │ [AI_Context]*.txt          │
                │ ─ Master_Data_L3│    └────────────────────────────┘
                └─────────────────┘

1.3 Component inventory

Component Path Role
Server entrypoint start_ns_online.py / stop_ns_online.py Daemonizes / kills ns_web_start.py cross-platform; manages PID file
Flask app network-sketcher_online/ns_web_start.py HTTPS server, request validation, security headers, daily access log, heartbeat eviction, parallel SVG generator
Engine bootstrap network-sketcher_online/ns_engine/nsm_adapter.py Headless Tk stubs (_Noop, Entry, StringVar...), openpyxl .nsm-load monkey-patch, in-process CLI runner (run_cli, run_cli_isolated)
Subprocess worker network-sketcher_online/ns_engine/_ns_worker.py Stand-alone Python that bypasses the in-process _cli_lock for true parallelism
CLI dispatcher network-sketcher_online/ns_engine/nsm_cli.py (Online), network-sketcher_offline/ns_cli.py (Offline) add / delete / rename / show / export verb routing, sub-command argument parsing
I/O adapter network-sketcher_online/ns_engine/nsm_io.py Reads/writes the 11 master sections from either .xlsx or .nsm (Parquet); type-preserves Python primitives via single-byte prefixes
L1 pipeline nsm_l1_diagram_create.py, nsm_l1_master_create.py, nsm_l1_table_from_master.py, nsm_l1_table_sync_master.py, nsm_l1_svg_create.py, nsm_l1_diagram_sync_master.py Layer-1 (physical wiring) diagram pipeline
L2 pipeline nsm_l2_diagram_create.py, nsm_l2_table_from_master.py, nsm_l2_table_sync_master.py, nsm_l2_svg_create.py Layer-2 (broadcast domain) per-area pipeline
L3 pipeline nsm_l3_diagram_create.py, nsm_l3_table_from_master.py, nsm_l3_table_sync_master.py, nsm_l3_svg_create.py Layer-3 (IP / VRF / VPN) pipeline
PPTX renderer nsm_ddx_figure.py python-pptx shape, line, segment, tag emitter
SVG renderers nsm_ddx_svg.py, nsm_ddx_svg_l2.py, nsm_ddx_svg_l3.py XML emitters for L1, L2, L3 SVG
Cross-layer sync nsm_sync_between_layers.py Keeps L1/L2/L3 tables consistent after a mutation
Extensions nsm_extensions.py AI-context export, flow report, IP report, auto-IP-addressing, summary diagram, CLI-on-GUI
Offline GUI network-sketcher_offline/network_sketcher.py Tkinter desktop application
External-system import network-sketcher_offline/ns_option_convert_to_master.py CML YAML, Visio SVG, draw.io SVG, NetBox CSV → master file
VPN diagrams network-sketcher_offline/ns_vpn_diagram_create.py L2 VPN overlay onto L1 (Offline only)

1.4 Technology stack

Layer Tooling
Language Python 3.9+ (3.11+ recommended)
Web framework Flask (single-app, no blueprints)
TLS Python ssl module with cipher pinning; self-signed cert generated via cryptography
Spreadsheet I/O openpyxl (read/write) + pandas (intermediate frames)
Compact storage pyarrow Parquet inside a ZIP (.nsm)
Presentation output python-pptx
Graph layout networkx for shortest-path / connected-components on flow-report and L2 broadcast-domain code paths
Numerics numpy
Concurrency concurrent.futures.ThreadPoolExecutor, subprocess.run (true parallel via separate interpreters), threading.Lock

1.5 Data flow for the canonical "upload + render" workflow

1. POST /upload (multipart) ──► validate_master_filename()
                              ► assign uuid4 job_id
                              ► save to uploads/<job_id>/[MASTER]*.xlsx
                              ► xlsx_to_nsm() → uploads/<job_id>/[MASTER]*.nsm
                              ► touch_heartbeat(job_id)
                              ◄ {job_id, filename}

2. POST /generate_step/<job_id>/init ──► run `show area`, `show device`,
                                          `show l1_link`, `show attribute`
                                          via run_ns_command()
                                       ► persist meta to .meta.json
                                       ◄ {areas, device_count, link_count,
                                          attribute_titles}

3. GET /svg_grid_stream/<job_id>     ──► fan out to ThreadPoolExecutor:
   (Server-Sent Events)                  - export l1_diagram --type all_areas_tag
                                          - export l1_diagram --type per_area_tag
                                          - export l3_diagram --type all_areas
                                          - export l3_diagram --type per_area
                                          - export l2_diagram --area <each>
                                       ► each task → run_ns_command_subprocess()
                                          (separate Python interpreter)
                                       ► moved files cached in _svg_mem_cache
                                       ◄ stream `data: {cell, files, svg_b64?}`
                                          for each completed cell
                                       ◄ background AI-context generation
                                          completes in parallel
                                       ◄ `data: {done: true, ai_filename}`

4. GET /download/<job_id>/<file>      ──► serve from uploads/<job_id>/

5. (heartbeat_timeout seconds idle)   ──► _cleanup_loop() rmtree(uploads/<job_id>/)
                                       ► drop _svg_mem_cache[job_id]
                                       ► drop _svg_grid_last_attr[job_id]

Note

The engine is deterministic. A given master file at a given commit always produces the same byte-for-byte output. This is intentional: diff-friendly diagrams let teams review topology changes the same way they review source code, and it's why subprocess timeouts default to a generous four hours rather than failing fast — large topologies are bounded by RAM and CPU, not by stochastic behavior.


2. System Requirements

2.1 Minimum

Resource Floor
CPU 2 cores (any x86_64 / ARM64 server-class CPU)
RAM 4 GB
Disk 1 GB free (source + dependencies + room for one session of upload artifacts)
Python 3.9 (CPython)
OS Linux (any modern distro), macOS 12+, Windows 10/11/Server 2019+
Network TCP 5443 inbound (configurable)
Resource Recommendation
CPU 4–8 cores. parallel_limit defaults to 6 and benefits from real cores up to that count
RAM 8–16 GB. L3 all_areas rendering for very large fabrics is memory-intensive
Disk 10 GB free. Per-session uploads/<job_id>/ directories can hold dozens of artifacts each
Python 3.11+ for the speed-up over 3.9 in pandas / pyarrow paths
OS Linux preferred (heartbeats and cleanup loop have been validated longest there)

2.3 Python package floor

The exact pinned versions live in network-sketcher_offline/requirements_offline.txt and network-sketcher_online/requirements_online.txt. The functional floor:

Package Why
flask Online server only
openpyxl All editions (Excel I/O)
pandas, pyarrow All editions (.nsm Parquet sections, I/O frames)
python-pptx All editions (PowerPoint emitter)
numpy All editions (layout math)
networkx All editions (broadcast-domain detection, flow shortest-paths)
cryptography Online edition (self-signed cert generation)
tkinter, tkinterdnd2 Offline GUI only

Important

The Online edition does not require Tk. nsm_adapter.bootstrap() injects no-op stubs into sys.modules['tkinter'], sys.modules['tkinter.ttk'], sys.modules['tkinter.filedialog'], and sys.modules['tkinter.messagebox'] before any engine module imports. This is how the Online edition runs cleanly in container images that have no display server.


3. Configuration Reference

This is the complete inventory of every configurable surface — JSON file, environment variable, and CLI flag. If a setting is not in one of these tables, it does not exist.

3.1 ns_web_config.json — the only config file

Located at network-sketcher_online/ns_web_config.json. Each top-level key is {"value": ..., "description": "..."}; the loader strips the description and keeps only the value. Keys whose names start with underscore (_description) are ignored.

Key Type Default Description
version string 3.0.1h Display version; not used by code, kept for human inspection
host string localhost Bind address. 0.0.0.0 for all NICs, or pin a specific IP
fqdn string | null null Certificate Common Name. If null, falls back to host
port int 5443 HTTPS listen port
max_file_size_mb int 50 Upload size cap (Flask MAX_CONTENT_LENGTH); over-cap returns HTTP 413
subprocess_timeout int | null 14400 Per-subprocess wall clock cap in seconds; null = no limit
heartbeat_timeout int 300 Seconds of inactivity before a session's uploads/<job_id>/ is wiped
cleanup_interval int 60 Seconds between idle-sweep runs in _cleanup_loop
parallel_limit int 6 max_workers for the SSE-driven SVG generator
ssl_cert_path string Certs/server.crt TLS cert path; relative to the config file or absolute
ssl_key_path string Certs/server.key TLS private key path
ssl_cert_validity_days int 3650 Validity window for auto-generated self-signed certs (10 years)
ssl_key_size int 4096 RSA key bits for auto-generated certs
ssl_min_version string TLSv1_2 One of TLSv1_2, TLSv1_3
ssl_max_version string TLSv1_3 One of TLSv1_2, TLSv1_3
ssl_ciphers string ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM OpenSSL cipher string
log_level string INFO One of DEBUG, INFO, WARNING, ERROR
access_log_retention_days int 90 Days of access_*.log to keep (purged on date roll-over and at startup)
ai_context_button_label string Copy & Open LLM UI button label (templated into HTML at request time)
ai_context_button_url string https://claude.ai URL the Copy & Open LLM button opens
ai_context_show_commands array<string> (15 show ... commands) Show-command list the AI Context exporter runs in parallel — the order is preserved in the output file. Default contains show area, show area_device, show area_location, show attribute, show device, show device_interface, show device_location, show l1_interface, show l1_link, show l2_broadcast_domain, show l2_interface, show l3_broadcast_domain, show l3_interface, show waypoint, show waypoint_interface

Important

ssl_cert_validity_days defaults to 3650 (10 years). For production, replace the auto-generated cert with one from your internal CA — most browsers cap manually-trusted self-signed certs at 1 year of acceptance. See Section 7.4.

3.2 Environment variables

Variable Read in Default Effect
NS_WEB_SERVER nsm_l2_diagram_create.py line 47 (Online and Offline copies) unset When set to any non-empty string, engine code follows the headless / server code path. The Online server sets it explicitly in ns_web_start.py.
NS_DEBUG_L3 nsm_l3_diagram_create.py line 324 (Online and Offline copies) 0 Set to 1 to dump verbose L3-layout diagnostics to stdout. Useful for diagnosing L3 layout regressions; ship the captured stdout when filing issues.

3.3 CLI flags (every ns_cli.py invocation)

3.3.1 Top-level verbs

Verb Required Notes
show <object> --master <path> Pure read. Output is a Python-array literal, suitable for ast.literal_eval
add <object> ... --master <path> Mutates the master. The Online server adds --skip_sync for all but the last syncable command in a batch
delete <object> ... --master <path> Mutates the master
rename <object> ... --master <path> Mutates the master; preserves all back-pointers
export <object> ... --master <path> Produces output artifacts (PPTX / SVG / Excel / TXT)

3.3.2 export l1_diagram flags

Flag Values Default Effect
--master path (required) Path to [MASTER]*.xlsx or [MASTER]*.nsm
--type all_areas, all_areas_tag, per_area, per_area_tag all_areas_tag Single combined view vs. one diagram per area; _tag variants include interface name labels
--attribute string (none) Title from the master file's Attribute sheet to color-code devices by
--format pptx, svg pptx Output format

3.3.3 export l2_diagram flags

Flag Values Default Effect
--master path (required) Master file
--area string (required) Area name (must exist in Master_Data_L2)
--format pptx, svg pptx Output format

3.3.4 export l3_diagram flags

Flag Values Default Effect
--master path (required) Master file
--type all_areas, per_area all_areas Single combined view vs. one diagram per area
--attribute string (none) Color-coding attribute title
--format pptx, svg pptx Output format

3.3.5 Other export subcommands

Subcommand Flags Effect
device_file --master Emits [DEVICE]<basename>.xlsx with normalized L1/L2/L3/Attribute sheets
ai_context_file --master, --accept-security-risk Emits [AI_Context]<basename>.txt. Requires either interactive YES confirmation or --accept-security-risk
master_file_backup --master Timestamped backup of the master file
master_file_nodata --master Empty-template master next to the input

3.3.6 Mutating-command behavior flags

Flag Read by Effect
--skip_sync All add/delete mutating subcommands Skips the post-mutation L2/L3 sync. Used internally by the Online server to amortize sync cost across a batch — only the last syncable command in a run executes the sync
--one_msg All show subcommands Concatenates output as a single string (rather than a list of lines). Used by the AI-context exporter to capture each show command's output in one piece

3.4 First-run-state files

File Created by Purpose
ns_online.pid start_ns_online.py PID of the running server; consumed by stop_ns_online.py
Certs/server.crt, Certs/server.key ns_web_start.py startup Self-signed TLS cert auto-generated if missing
logs/access_YYYYMMDD.log _DailyFileHandler One file per local date; /heartbeat/* paths excluded
logs/server.log start_ns_online.py Detached stdout/stderr
uploads/<uuid4>/ /upload route Per-job working directory; auto-deleted on heartbeat expiry
uploads/<uuid4>/.active_master set_active_master() Pointer to the currently-active master filename within the job dir (handles versioning)
uploads/<uuid4>/.meta.json /generate_step/init Cached {areas, device_count, link_count, attribute_titles} for /restore

4. Deployment & Scaling

Network Sketcher Online is a single-process Flask app — there is no master/worker pool, no separate cache process, no message broker. This keeps deployment simple but bounds horizontal scaling to running more than one instance behind a load-balancer with sticky sessions (necessary because per-session state lives in uploads/<job_id>/ on the local filesystem).

4.1 Single-host deployment (typical)

# 1. Clone and install
git clone https://github.com/cisco-open/network-sketcher.git
cd network-sketcher
python -m pip install -r network-sketcher_online/requirements_online.txt

# 2. Optional: edit ns_web_config.json for your host/port
${EDITOR:-vi} network-sketcher_online/ns_web_config.json

# 3. Start
python start_ns_online.py

The server runs in the foreground of the Python process started by Popen, but start_ns_online.py detaches it on Windows (DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP) and creates a new POSIX session on Linux/macOS (start_new_session=True).

4.2 Run as a systemd unit (Linux)

# /etc/systemd/system/network-sketcher.service
[Unit]
Description=Network Sketcher Online
After=network.target

[Service]
Type=forking
User=netsketcher
WorkingDirectory=/opt/network-sketcher
ExecStart=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/start_ns_online.py
ExecStop=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/stop_ns_online.py
PIDFile=/opt/network-sketcher/network-sketcher_online/ns_online.pid
Restart=on-failure

[Install]
WantedBy=multi-user.target

Tip

Use a dedicated unprivileged user for the service. The Online server never needs root; reading the config and binding port 5443 work fine as a normal user (ports above 1024).

4.3 Containerized deployment

FROM python:3.11-slim

WORKDIR /opt/network-sketcher
COPY . .
RUN pip install --no-cache-dir \
    -r network-sketcher_online/requirements_online.txt

# Persist Certs and uploads across container restarts
VOLUME ["/opt/network-sketcher/network-sketcher_online/Certs",
        "/opt/network-sketcher/network-sketcher_online/uploads",
        "/opt/network-sketcher/network-sketcher_online/logs"]

EXPOSE 5443
CMD ["python", "start_ns_online.py"]

4.4 Reverse proxy with TLS termination

If you front Network Sketcher with nginx/Caddy/Traefik that already holds your real cert:

  1. Set host: 127.0.0.1 and let the reverse proxy bind public ports.
  2. Provide the upstream's own cert via ssl_cert_path / ssl_key_path, or disable upstream TLS by replacing the ssl.SSLContext in ns_web_start.py with a plain HTTP listener (currently not exposed as a config flag — patch required).
  3. Forward X-Forwarded-For so access logs report client IPs.

Note

The default config bakes HSTS into every response with max-age=31536000; includeSubDomains. Remove that header if you're intentionally serving HTTP behind a TLS-terminating proxy without HTTPS upstream.

4.5 Horizontal scaling

Each session's working files live on local disk under uploads/<job_id>/. To run more than one Online instance:

  • Place a load balancer in front (HAProxy / nginx / your cloud LB).
  • Configure sticky sessions keyed on a cookie or header so a given job_id always lands on the same backend.
  • Keep Certs/, uploads/, and logs/ on instance-local volumes — do not put uploads/ on shared NFS unless you're willing to serialize all parallel writes.

4.6 Stopping cleanly

stop_ns_online.py finds every running ns_web_start.py process via the OS process list (Windows: wmic process where ..., Linux/macOS: pgrep -f ns_web_start.py) and falls back to the PID file as a secondary source. It sends SIGTERM (POSIX) or taskkill /F /T /PID (Windows), then waits up to 10 seconds for the process(es) to exit.

Warning

A SIGTERM during diagram generation aborts the in-flight subprocess. Sessions persist on disk and the user can re-issue the same generate request after the server is back up — but any partial output files in the job directory are not cleaned automatically (the next session's heartbeat-expiry sweep will eventually remove them).


5. Operations

5.1 Health check

There is no dedicated /health route. A useful liveness probe is the root path:

curl -ksf https://127.0.0.1:5443/ -o /dev/null && echo OK

A readiness probe should additionally verify the cert paths exist and are readable.

5.2 Daily access log

logs/access_YYYYMMDD.log (one file per local date) contains one line per request:

2026-04-28 14:07:31 198.51.100.42 POST /upload 200 upload="[MASTER]campus.xlsx"
2026-04-28 14:07:33 198.51.100.42 POST /generate_step/96d6b2a4-.../init 200
2026-04-28 14:07:35 198.51.100.42 GET /svg_grid_stream/96d6b2a4-... 200
2026-04-28 14:08:12 198.51.100.42 GET /download/96d6b2a4-.../[L1_DIAGRAM]AllAreasTag_campus.pptx 200 download="[L1_DIAGRAM]AllAreasTag_campus.pptx"

/heartbeat/<job_id> is suppressed from access logs (it would otherwise fire every few seconds). Files older than access_log_retention_days are deleted on startup and on each date roll-over.

Note

Network Sketcher does not rotate or compress access logs itself. If you ship them off-host, run logrotate (or the equivalent) on logs/access_*.log; the in-app cleanup is best-effort, not a backup.

5.3 Heartbeat eviction

Every successful upload registers a heartbeat with touch_heartbeat(). The browser front-end refreshes it every few seconds via POST /heartbeat/<job_id>. The server's _cleanup_loop thread runs every cleanup_interval seconds and rmtree-s any session whose heartbeat is older than heartbeat_timeout. The same loop drops the SVG memory cache and the per-session attribute cache.

5.4 Common operational checklists

After a clean install

  • [ ] python start_ns_online.py starts without error
  • [ ] https://localhost:5443/ loads in a browser
  • [ ] Upload a master file, wait for the SSE grid to populate
  • [ ] Confirm logs/access_YYYYMMDD.log is being written
  • [ ] Confirm Certs/server.crt was created (4096-bit RSA, 10-year validity)

After a config change

  • [ ] python stop_ns_online.py reports Stopped N process(es).
  • [ ] python start_ns_online.py reports a fresh PID
  • [ ] First request after restart returns 200
  • [ ] Loaded parallel_limit, port, etc. match what's now in ns_web_config.json

Before a TLS cert renewal

  • [ ] Backup Certs/server.crt and Certs/server.key
  • [ ] Drop in the new cert and key with the same paths (or update ssl_cert_path / ssl_key_path)
  • [ ] Restart
  • [ ] Confirm the new fingerprint with openssl x509 -in Certs/server.crt -noout -fingerprint -sha256

5.5 Backup strategy

What Why How often
network-sketcher_online/ns_web_config.json Config drift recovery Whenever changed
network-sketcher_online/Certs/ Avoid forced re-trust on rebuild Whenever cert rotates
Source repo at the running commit Reproducibility At deploy time
Master files ([MASTER]*.xlsx) and their backups Topology data is the only irreplaceable artifact Daily, off-host

uploads/<job_id>/ directories are not backup-worthy — they're transient and auto-deleted.

5.6 Frequent cross-edition syncs

Both editions duplicate ~95% of the engine code (Offline ns_*.py and Online ns_engine/nsm_*.py). When you backport a fix:

  1. Patch the Online file under ns_engine/.
  2. Diff against the Offline file under network-sketcher_offline/ — the names match one-to-one (ns_l2_diagram_create.py ↔ nsm_l2_diagram_create.py).
  3. Apply the corresponding change to the Offline file.
  4. Run python ns_cli.py show area --master "[MASTER]demo.xlsx" from each edition to confirm dispatcher integrity.

6. Performance

6.1 Where time goes

For a "first render after upload":

Phase Driver Typical share
xlsx_to_nsm conversion Master file size, sheet count 5–15%
show area/device/l1_link/attribute (init) Master file size 5%
L1 all_areas_tag diagram Device + link count, area count 20–30%
L1 per_area_tag (one per area) Per-area complexity 10–20%
L3 all_areas diagram Device + L3 interface count 25–40%
L3 per_area Per-area L3 density 5–15%
L2 per area L2 broadcast-domain density 5–15% (× number of areas)
AI Context (parallel) Master file size runs concurrently with diagrams

/svg_grid_stream parallelizes across parallel_limit workers; the critical path is the slowest single subprocess. For most fabrics that's the L3 all_areas render.

6.2 Tuning levers

Lever Effect When to use
Raise parallel_limit More concurrent diagrams You have spare cores and RAM
Lower parallel_limit Less RAM pressure You're hitting OOMs / multiprocessing fallback errors
Raise subprocess_timeout (or null) Bigger fabrics finish You've sized RAM for the job and don't want timeouts
Lower heartbeat_timeout Frees disk faster Many short sessions, disk pressure
Raise heartbeat_timeout Better UX for slow users Few long sessions
Use .nsm directly Skips xlsx→nsm conversion Repeated renders against an unchanged master
Use --type per_area Avoids the all_areas blow-up on huge fabrics Hundreds of devices, dozens of areas

6.3 SVG caching path

After a successful subprocess render, the moved file's bytes are cached in _svg_mem_cache[job_id][filename] under _svg_mem_cache_lock. Subsequent requests to /svg_raw/<job_id>/... serve from this cache, bypassing both the disk and the Windows AV-lock re-read. The cache is dropped on heartbeat expiry.

6.4 Known performance hazards

Warning

On Windows, antivirus engines (notably Defender) re-scan files immediately after they're moved between directories. The engine mitigates this with _wait_readable() (10-second wait with 150ms polling) before serving SVGs. If you see "File not readable after 10s" warnings in server.log, your AV is holding files longer than 10 seconds — exclude the uploads/ directory from real-time scanning or raise the timeout in _wait_readable.


7. Security

Network Sketcher's outputs are sensitive. A network topology artifact reveals device names, vendor inventory (via interface naming), IP plans, VRF / VPN structure, and broadcast-domain layout. Treat every generated file as Internal at minimum, and the AI Context file as Confidential.

7.1 What's in the box at first start

Defense Implementation Notes
TLS by default Auto-generated 4096-bit RSA self-signed cert with 10-year validity Replace with a real cert before exposing beyond localhost
TLS version pinning ssl_min_version: TLSv1_2, ssl_max_version: TLSv1_3 TLS 1.0/1.1 disabled by default
Cipher pinning ssl_ciphers: ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM No RC4, no DES, no NULL
Strict-Transport-Security max-age=31536000; includeSubDomains Forces browsers to HTTPS for one year
X-Frame-Options DENY (default), SAMEORIGIN for /device_preview/<...> Prevents clickjacking; allows internal thumbnail iframes
Content-Security-Policy default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; ...; frame-ancestors 'none' XSS / data-exfil reduction
X-Content-Type-Options nosniff Stops MIME sniffing exploits
Referrer-Policy strict-origin-when-cross-origin Limits referrer leakage
Job ID validation UUID4 regex ^[a-f0-9\-]{36}$ Prevents path traversal via job id
Filename validation Must start with [MASTER] and end in .xlsx or .nsm Rejects upload-traversal attempts
Upload size cap max_file_size_mb (default 50 MB) Returns HTTP 413 over cap
Per-session isolation UUID4 directories One user can't read another's master file

Caution

The default deployment has no authentication. Anyone who can reach the listening port can upload, render, and download. Never expose a default Network Sketcher Online directly to the internet. Put it behind an auth proxy (oauth2-proxy, Pomerium, your SSO IdP), a VPN, or a Tailscale tailnet before exposing it beyond localhost.

7.2 Sensitive data classification

Artifact Sensitivity Why
[MASTER]*.xlsx / *.nsm High Contains the entire network plan
[L1_DIAGRAM]* / [L2_DIAGRAM]* / [L3_DIAGRAM]* High Visualizes device names, interface IDs, VLAN/VRF mappings
[DEVICE]*.xlsx High Tabular form of all interfaces and IPs
[AI_Context]*.txt Highest Plain-text dump intended for an external LLM
logs/access_*.log Medium Source IPs and request paths but no payloads
logs/server.log Medium May contain stack traces with file paths

Warning

The AI Context file is the highest-risk output. It is designed to be pasted into a hosted LLM, which means it will be transmitted to a third party and may be retained for training (per the LLM vendor's TOS). The CLI requires either an interactive YES confirmation or --accept-security-risk; the web UI displays the same risk text in the tooltip. Audit any policy that allows unattended --accept-security-risk use. See nsm_extensions.py for the export logic and the warning text.

Role Should access Should not access
Network engineer (read) The web UI for their own organization's master files; downloaded diagrams under their own handling Other tenants' master files; raw uploads/ directory on the host
Network engineer (write) The web UI to mutate their own master via the CLI Commands panel The host filesystem directly; other engineers' active sessions
Sysadmin / SRE The host filesystem (/opt/network-sketcher), logs/, Certs/, ns_web_config.json Master file contents (can read them, but should treat as confidential by policy)
External / 3rd-party Nothing (use a separate instance per tenant, or strong front-door auth) All of the above

7.4 Production TLS

The auto-generated cert is for development convenience. To replace it:

# Stop the server
python stop_ns_online.py

# Drop in CA-issued cert and key
cp /path/to/your/cert.pem  network-sketcher_online/Certs/server.crt
cp /path/to/your/key.pem   network-sketcher_online/Certs/server.key

# Or set custom paths in ns_web_config.json:
#   "ssl_cert_path": {"value": "/etc/letsencrypt/live/sketcher.example.com/fullchain.pem"}
#   "ssl_key_path":  {"value": "/etc/letsencrypt/live/sketcher.example.com/privkey.pem"}
#   "fqdn":          {"value": "sketcher.example.com"}

python start_ns_online.py

Verify:

openssl s_client -connect sketcher.example.com:5443 -servername sketcher.example.com < /dev/null 2>&1 | openssl x509 -noout -subject -dates

7.5 Secrets handling

Network Sketcher itself stores no secrets in code or config. The only sensitive material on disk is:

  • TLS private key (Certs/server.key) — chmod 0600, owned by the service user.
  • Master files in uploads/<job_id>/ — auto-cleaned on heartbeat expiry. For long-lived hosting, consider mounting uploads/ on an encrypted volume.

There are no API keys, no database credentials, no third-party tokens. The ai_context_button_url defaults to a public LLM URL but is purely a hyperlink — Network Sketcher itself never calls out to it.

7.6 Sanitization recommendations before sharing diagrams

If you need to share a diagram externally (vendor RFP, conference talk, public docs):

  1. Make a copy of the master and run a rename pass:
    • rename device <real-name> <Device-N> for every device
    • rename area <real-area> <Area-N> for every area
  2. Run a delete ip_address pass on the L3 sheet (or set IPs to RFC1918-only representatives via add ip_address).
  3. Re-render the diagrams from the sanitized master.
  4. Do not redact PNG/PPTX exports manually — the underlying SVG may still contain the un-redacted text in <text> elements.

8. Developer Internals

8.1 Repository layout

network-sketcher/
├── README.md                  # Cisco Open project README
├── CONTRIBUTING.md
├── CODE_OF_CONDUCT.md
├── SECURITY.md
├── LICENSE                    # Apache 2.0
├── start_ns_online.py         # Cross-platform daemonizer for Online
├── stop_ns_online.py          # Cross-platform stop
├── network-sketcher_offline/  # Tkinter GUI + standalone CLI
│   ├── network_sketcher.py    # GUI entrypoint
│   ├── ns_cli.py              # Standalone CLI dispatcher
│   ├── ns_def.py              # Shared helpers (file I/O, sheet→array, etc.)
│   ├── ns_dev.py              # GUI internals
│   ├── ns_extensions.py       # AI context, flow report, IP report, etc.
│   ├── ns_l1_*.py             # L1 pipeline (5 files)
│   ├── ns_l2_*.py             # L2 pipeline (3 files)
│   ├── ns_l3_*.py             # L3 pipeline (3 files)
│   ├── ns_ddx_figure.py       # PPTX renderer
│   ├── ns_egt_maker.py        # Excel template maker
│   ├── ns_attribute_table_*.py
│   ├── ns_option_convert_to_master.py  # External-system imports
│   ├── ns_sync_between_layers.py
│   ├── ns_vpn_diagram_create.py
│   ├── hook-pptx.py           # PyInstaller hook
│   ├── ns_logo.png
│   ├── requirements_offline.txt
│   └── Sample_data/
└── network-sketcher_online/   # Flask server + headless engine
    ├── ns_web_start.py        # Flask app: routes, security headers, parallelism
    ├── ns_web_config.json     # The only config file
    ├── help_msg.json          # Tooltip text for the web UI
    ├── requirements_online.txt
    ├── Certs/                 # Auto-populated on first start
    ├── logs/                  # Auto-populated on first request
    ├── static/                # JS deps (PptxViewJS, chart.umd, jszip, xlsx)
    └── ns_engine/             # Headless engine (mirror of offline ns_*)
        ├── __init__.py
        ├── _ns_worker.py      # Subprocess worker entry
        ├── nsm_adapter.py     # Tk stubs + bootstrap + run_cli
        ├── nsm_io.py          # xlsx ↔ nsm (Parquet) bridge
        ├── nsm_cli.py
        ├── nsm_def.py
        ├── nsm_l1_*.py / nsm_l2_*.py / nsm_l3_*.py
        ├── nsm_ddx_figure.py / nsm_ddx_svg*.py
        ├── nsm_extensions.py
        ├── nsm_svg_compute.py / nsm_svg_worker.py
        └── nsm_sync_between_layers.py

8.2 Key invariants

Note

Master file integrity is preserved by design. The Online server never overwrites an uploaded master in place: any mutation creates a versioned copy ([MASTER]<basename>_<n>.xlsx) and updates .active_master. This is the single most important behavior to preserve when patching.

Note

The engine is deterministic with respect to a given master + flag combination. If a refactor changes byte-for-byte output, that's a regression that needs review even if the diagrams look the same. Reviewers should diff the SVG output against a known-good master in network-sketcher_offline/Sample_data/.

8.3 Module reference

ns_web_start.py — Flask app

Major helpers:

Helper Purpose
_load_config() Loads ns_web_config.json, strips description, returns flat dict
_load_help_messages() Loads help_msg.json for the UI tooltip system
validate_master_filename(filename) Enforces [MASTER]*.xlsx/*.nsm
sanitize_job_id(job_id) Enforces UUID4 regex
find_master_file(work_dir) Picks the first [MASTER]* in a job dir
get_active_master(work_dir) / set_active_master(work_dir, name) .active_master pointer
run_ns_command(args) In-process CLI run via nsm_adapter.run_cli
run_ns_command_isolated(args, work_dir, master_filename) Same, but in a temp subdir copy
run_ns_command_subprocess(args, work_dir, master_filename) Separate Python interpreter (true parallelism)
_wait_readable(path, timeout=10.0) Windows AV-lock backoff
generate_ai_context_parallel(work_dir, master_filename) Fans out the configured show commands across a thread pool
_DailyFileHandler Per-day log file with auto-cleanup
_cleanup_loop() Heartbeat-driven session sweep
_convert_svg_for_visio(svg_bytes) SVG2 → SVG1.1 transformations for Visio
_convert_svg_to_drawio(svg_bytes) SVG → mxGraphModel XML

nsm_adapter.py — engine bootstrap

Component Purpose
_Noop Catch-all stub class returning self for any attribute / call. Backs every Tk widget stub
Toplevel, Frame, Label, Entry, Text, StringVar, IntVar, BooleanVar, ... One stub class per Tk widget the engine references
_TtkStub Equivalent stubs for tkinter.ttk
_FileDialogStub askopenfilename, askopenfilenames, asksaveasfilename, askdirectory (all return empty)
_MessageBoxStub showinfo, showwarning, showerror, askquestion, askokcancel, askyesno, askretrycancel
bootstrap() Idempotent: injects all stubs into sys.modules, sets up sys.path, monkey-patches openpyxl.load_workbook to handle .nsm files
_patch_openpyxl_for_nsm() Transparent .nsm → temp-.xlsx translation, cached for the session
run_cli(args, cwd) In-process CLI runner with global _cli_lock (single command at a time per process)
run_cli_isolated(args, work_dir, master_filename) Temp subdir + master copy + run + move outputs

nsm_io.py — master file I/O

Helper Purpose
SECTION_SHEET_MAP The 11 master sections with their (sheet_name, section_tag)
_encode_cell(value) / _decode_cell(v) Type-preserving encoder using single-byte prefix (\x00I int, \x00F float, \x00B bool)
_read_section_from_xlsx(path, ws_name, section_tag) Section reader that locates <<TAG>> and walks until the next tag or 3000 empty rows
xlsx_to_nsm(xlsx_path, nsm_path) ZIP of Parquet files
nsm_to_xlsx(nsm_path, xlsx_path) Reverse

nsm_cli.py — verb dispatcher

The dispatcher reads argv_array and routes by literal verb match ('show' in argv_array, etc.), then by next_arg against the verb's allow-list. The five subcommand allow-lists are documented in Section 3.3. Any unsupported verb or object prints the supported list and exits.

Pipelines (nsm_l1_*.py, nsm_l2_*.py, nsm_l3_*.py)

Each layer has up to four files:

Pattern Role
nsm_lN_master_create.py (L1 only) Builds the per-layer master from the high-level inputs
nsm_lN_table_from_master.py Extracts the per-layer table view (shipped in [DEVICE]*.xlsx)
nsm_lN_table_sync_master.py Keeps the per-layer table view consistent after master mutations
nsm_lN_diagram_create.py Lays out the diagram and emits PPTX
nsm_lN_svg_create.py Same layout, SVG emitter
nsm_l1_diagram_sync_master.py (L1 only) Re-positions wires after device/area moves

nsm_extensions.py — feature catalog

Class Purpose
ai_context.export_ai_context_file(self, dummy) Writes [AI_Context]<basename>.txt from 15 show commands plus nsm_extensions_cmd_list.txt
flow_report.create_device_flow_table(self, full_filepath_master, device_name) Per-device flow table with shortest-path routing (networkx)
flow.add_routing_path_to_flow(...) / .get_shortest_path(...) / .append_flows_to_diagram(...) / .get_flow_item_list(...) / .export_flow_file(...) Flow-report internals
ip_report.export_ip_report(self) / .get_folder_list(self) IP allocation report
auto_ip_addressing.run_auto_ip(...) / .get_folder_list(...) / .get_auto_ip_param(...) Auto-assign IPs from a parameter set
summary_diagram.export_summary_diagram(self) Single-slide topology summary
cli_on_gui.execute_cli_commands(...) / .show_cli_input_window(...) / .show_command_output_window(...) Offline GUI CLI panel

8.4 Code patterns and conventions

  • Tk in the engine? All Tk imports go through stubs at module load via nsm_adapter.bootstrap(). The engine still calls tk.Entry(), tk.StringVar() etc. — those resolve to the _Noop-backed classes. Don't try to remove the Tk imports; they're required for shared code with the Offline edition.
  • CLI verb parsing: simple membership checks ('show' in argv_array) followed by get_next_arg(...) lookups. There's no argparse use; if you add a flag, follow the existing if a == '--master': replace_next = True pattern.
  • Versioning: never overwrite the input. Mutations write to [MASTER]<basename>_<n>.xlsx and update .active_master.
  • Headless detection: check os.environ.get('NS_WEB_SERVER') rather than detecting Tk presence — the stubs make Tk presence always-truthy in Online mode.
  • Subprocess vs. in-process: the Online server uses three execution modes: run_ns_command (in-process, holds _cli_lock), run_ns_command_isolated (in-process with master copy), run_ns_command_subprocess (separate interpreter for true parallelism). Pick the lowest-overhead one that satisfies your isolation needs.

8.5 Testing strategy

The repository ships no formal test suite. The maintainer-tested verification path is:

  1. python ns_cli.py show area --master Sample_data/... — sanity check that the dispatcher and master loader are wired up.
  2. python ns_cli.py export l1_diagram --master Sample_data/... --type all_areas_tag — full L1 pipeline.
  3. python ns_cli.py export l2_diagram --master Sample_data/... --area <area> — full L2 pipeline per area.
  4. python ns_cli.py export l3_diagram --master Sample_data/... --type all_areas — full L3 pipeline.
  5. End-to-end via the web UI for a final visual check.

When contributing, run all four CLI exports against Sample_data/No_data_Master_file_with_AI_Context/[MASTER]no_data.xlsx and visually diff the resulting .pptx against the previous version.

Tip

Sample data is intentionally minimal — it's a no-data master with the AI-context placeholder. For more thorough testing, build a small synthetic master with two areas, four devices, and a handful of L1 wires; commit it locally as a fixture.

8.6 Potential improvements

  • Test suite. Even a smoke-test wrapping the four CLI exports would catch most regressions.
  • /health endpoint. Currently health checks must hit /.
  • Auth integration hook. The server has no auth surface; a pluggable middleware that delegates to JWT / OIDC / mTLS would make multi-tenant deploys safer.
  • Output retention policy. uploads/<job_id>/ currently lives for exactly heartbeat_timeout seconds; some users want "keep last 24 hours regardless of activity". Make the retention policy configurable separately from the heartbeat window.
  • Structured error response. Routes return mixed {success, message} and {error} shapes. Standardizing would help programmatic clients.
  • Headless Tk removal. Long-term, the engine would be cleaner if Tk references were stripped rather than stubbed. This is a large refactor (~25 modules) but enables packaging the engine as a library without the Adapter dance.

8.7 Contributing

The repository follows the Cisco Open contribution model:

  • All code is Apache 2.0; every file carries the SPDX header SPDX-License-Identifier: Apache-2.0 and Copyright 2023 Cisco Systems, Inc. and its affiliates.
  • Open issues at github.com/cisco-open/network-sketcher/issues.
  • See CONTRIBUTING.md and CODE_OF_CONDUCT.md at the repo root.
  • Security issues go to the address in SECURITY.md, not the public issue tracker.

9. Integration Points

Network Sketcher is intentionally narrow at its edges. It does not talk to controllers, doesn't push configs, doesn't poll devices. Integration happens through file imports and exports.

9.1 Inbound: external-system imports (Offline GUI)

The Offline GUI's External Systems tab imports four formats into a master file via ns_option_convert_to_master.py:

Format Source Class in code
YAML Cisco CML topology export ns_option_convert_to_master_yaml
SVG Visio export ns_option_convert_to_master_svg
SVG draw.io export ns_option_convert_to_master_svg (same class, dialect-detected)
CSV NetBox device export ns_option_convert_to_master_csv

Two helper classes round out the import flow:

  • ns_overwrite_line_to_master_yaml — overlays a YAML topology onto an existing master.
  • ns_l3_config_to_master_yaml — adds L3 details from a YAML config bundle.

9.2 Outbound: derived export formats

Format Trigger Implementation
.pptx export l1_diagram / l2_diagram / l3_diagram (default --format pptx) nsm_ddx_figure.py (python-pptx)
.svg --format svg on diagram exports nsm_l1_svg_create.py / nsm_l2_svg_create.py / nsm_l3_svg_create.py
Visio-friendly .svg GET /download_visio/<job_id>/<file> _convert_svg_for_visio() in ns_web_start.py
draw.io .drawio (XML) GET /download_drawio/<job_id>/<file> _convert_svg_to_drawio() in ns_web_start.py
[DEVICE]<basename>.xlsx export device_file Composed from L1/L2/L3/Attribute pipelines
[AI_Context]<basename>.txt export ai_context_file nsm_extensions.ai_context.export_ai_context_file()
.nsm (ZIP of Parquet) Auto on Online upload nsm_io.xlsx_to_nsm()
ZIP of all session artifacts GET /download_all/<job_id> Built in-memory by download_all()

9.3 LLM integration

The AI Context flow is the only outbound call path:

  1. The web UI (or nsm_extensions.ai_context.export_ai_context_file) builds a deterministic text bundle from the configured show commands plus nsm_extensions_cmd_list.txt.
  2. The user clicks Copy & Open LLM (or hits the Download button). The button label and target URL are configured via ai_context_button_label and ai_context_button_url (default https://claude.ai).
  3. The clipboard content is the AI Context plus the user's natural- language prompt.
  4. The user pastes into the LLM, gets back CLI commands, pastes them into the CLI Commands box, and hits Run.

Network Sketcher itself never sends the data anywhere — the user controls the transmission.

9.4 Programmatic engine usage

You can embed the engine in another Python program by bootstrapping once and calling run_cli() repeatedly:

from ns_engine.nsm_adapter import bootstrap, run_cli

bootstrap()

result = run_cli([
    'export', 'l1_diagram',
    '--master', '/path/to/[MASTER]campus.xlsx',
    '--type', 'all_areas_tag',
    '--format', 'svg',
])

print(result.returncode)   # 0 on success
print(result.stdout)        # captured stdout
print(result.stderr)        # captured stderr

run_cli holds a single global _cli_lock, so calls are serialized within one process. For true parallelism, follow the pattern in run_ns_command_subprocess and shell out to _ns_worker.py instead.

Tip

If you're embedding the engine, prefer run_cli_isolated() over run_cli() for any mutating command — it copies the master into a temp directory first, so a crash in your code can't corrupt the caller's master file.

9.5 Front-end JS dependencies

The Online edition's UI uses these vendored JS libraries (under network-sketcher_online/static/):

File Purpose
PptxViewJS.min.js In-browser PowerPoint preview (zoom, pan)
chart.umd.min.js Chart rendering for analytics panels
jszip.min.js Client-side ZIP support for previews
xlsx.full.min.js In-browser Excel preview
logo.png Brand asset

These are vendored to keep the deployment self-contained — no CDN calls happen at runtime. If you upgrade a library, replace the file in place and restart the server.