This guide is the operational and architectural reference for engineers who run Network Sketcher in shared environments, integrate it with surrounding systems, or extend it. It complements the User Guide: where the User Guide answers "how do I draw a diagram?", this guide answers "how does the engine work, what does every config knob do, what are the failure modes, and where do I patch behavior?".
Note
The repository ships two editions that share a common rendering engine. Operational concerns differ — the Online edition is what you deploy on a server; the Offline edition is what users run on their workstations. This guide is structured for the Online edition and calls out Offline differences explicitly where they matter.
1. Architecture Overview
Network Sketcher Online is a single-process Flask web service that
fronts a deterministic Python rendering engine. The engine is the same
code in both editions; the Online edition wraps it in headless stubs
(nsm_adapter.py) so it can run without a display server.
1.1 What it does
A user uploads a [MASTER]*.xlsx (or .nsm) file. The engine reads its
three well-known sheets (Master_Data, Master_Data_L2,
Master_Data_L3), decomposes them into in-memory section arrays, and
runs one of three dedicated diagram pipelines (L1, L2, L3) against them.
Output is written as PowerPoint (.pptx), SVG, or Excel
([DEVICE]*.xlsx) and offered for download. A CLI layer
(add/delete/rename/show/export) lets users mutate the master
file by issuing line-oriented commands; mutations are versioned so the
original is preserved.
1.2 ASCII architecture
┌─────────────────────────────────────────────────┐
│ Browser (HTTPS, :5443) │
│ Single-page UI · SSE · localStorage(job_id) │
└────────────────────┬────────────────────────────┘
│ TLS (self-signed by default)
┌────────────────────▼────────────────────────────┐
│ ns_web_start.py (Flask app) │
│ Routes: /upload /generate_step /run_commands │
│ /svg_grid_stream /download(_visio|_drawio|all)│
│ Headers: HSTS · X-Frame-Options · CSP │
│ Per-job state: uploads/<uuid>/ │
│ Heartbeats: in-memory dict, 5-min idle expiry │
└────┬───────────────┬───────────────┬────────────┘
│ │ │
in-process run_cli │ subprocess(_ns_worker.py)
│ │ │
┌────▼───────────────▼───────────────▼────────────┐
│ ns_engine/ (headless engine) │
│ nsm_adapter.py Tk stubs + bootstrap │
│ nsm_cli.py verb dispatcher (5 verbs) │
│ nsm_io.py .xlsx ↔ .nsm (Parquet) bridge │
│ nsm_l1_*.py L1 diagram + table + sync │
│ nsm_l2_*.py L2 diagram + table + sync │
│ nsm_l3_*.py L3 diagram + table + sync │
│ nsm_ddx_*.py pptx / svg renderers │
│ nsm_extensions.py AI-context · flow · IP report │
│ nsm_sync_between_layers.py L2↔L3 consistency │
└────┬───────────────────────┬─────────────────────┘
│ │
┌───────▼─────────┐ ┌────────▼───────────────────┐
│ Master file │ │ Output artifacts │
│ [MASTER]*.xlsx │ │ [L1_DIAGRAM]*.pptx/.svg │
│ or *.nsm (zip │ │ [L2_DIAGRAM]<area>_* │
│ of Parquet) │ │ [L3_DIAGRAM]* │
│ ─ Master_Data │ │ [DEVICE]*.xlsx │
│ ─ Master_Data_L2│ │ [AI_Context]*.txt │
│ ─ Master_Data_L3│ └────────────────────────────┘
└─────────────────┘
1.3 Component inventory
| Component | Path | Role |
|---|---|---|
| Server entrypoint | start_ns_online.py / stop_ns_online.py |
Daemonizes / kills ns_web_start.py cross-platform; manages PID file |
| Flask app | network-sketcher_online/ns_web_start.py |
HTTPS server, request validation, security headers, daily access log, heartbeat eviction, parallel SVG generator |
| Engine bootstrap | network-sketcher_online/ns_engine/nsm_adapter.py |
Headless Tk stubs (_Noop, Entry, StringVar...), openpyxl .nsm-load monkey-patch, in-process CLI runner (run_cli, run_cli_isolated) |
| Subprocess worker | network-sketcher_online/ns_engine/_ns_worker.py |
Stand-alone Python that bypasses the in-process _cli_lock for true parallelism |
| CLI dispatcher | network-sketcher_online/ns_engine/nsm_cli.py (Online), network-sketcher_offline/ns_cli.py (Offline) |
add / delete / rename / show / export verb routing, sub-command argument parsing |
| I/O adapter | network-sketcher_online/ns_engine/nsm_io.py |
Reads/writes the 11 master sections from either .xlsx or .nsm (Parquet); type-preserves Python primitives via single-byte prefixes |
| L1 pipeline | nsm_l1_diagram_create.py, nsm_l1_master_create.py, nsm_l1_table_from_master.py, nsm_l1_table_sync_master.py, nsm_l1_svg_create.py, nsm_l1_diagram_sync_master.py |
Layer-1 (physical wiring) diagram pipeline |
| L2 pipeline | nsm_l2_diagram_create.py, nsm_l2_table_from_master.py, nsm_l2_table_sync_master.py, nsm_l2_svg_create.py |
Layer-2 (broadcast domain) per-area pipeline |
| L3 pipeline | nsm_l3_diagram_create.py, nsm_l3_table_from_master.py, nsm_l3_table_sync_master.py, nsm_l3_svg_create.py |
Layer-3 (IP / VRF / VPN) pipeline |
| PPTX renderer | nsm_ddx_figure.py |
python-pptx shape, line, segment, tag emitter |
| SVG renderers | nsm_ddx_svg.py, nsm_ddx_svg_l2.py, nsm_ddx_svg_l3.py |
XML emitters for L1, L2, L3 SVG |
| Cross-layer sync | nsm_sync_between_layers.py |
Keeps L1/L2/L3 tables consistent after a mutation |
| Extensions | nsm_extensions.py |
AI-context export, flow report, IP report, auto-IP-addressing, summary diagram, CLI-on-GUI |
| Offline GUI | network-sketcher_offline/network_sketcher.py |
Tkinter desktop application |
| External-system import | network-sketcher_offline/ns_option_convert_to_master.py |
CML YAML, Visio SVG, draw.io SVG, NetBox CSV → master file |
| VPN diagrams | network-sketcher_offline/ns_vpn_diagram_create.py |
L2 VPN overlay onto L1 (Offline only) |
1.4 Technology stack
| Layer | Tooling |
|---|---|
| Language | Python 3.9+ (3.11+ recommended) |
| Web framework | Flask (single-app, no blueprints) |
| TLS | Python ssl module with cipher pinning; self-signed cert generated via cryptography |
| Spreadsheet I/O | openpyxl (read/write) + pandas (intermediate frames) |
| Compact storage | pyarrow Parquet inside a ZIP (.nsm) |
| Presentation output | python-pptx |
| Graph layout | networkx for shortest-path / connected-components on flow-report and L2 broadcast-domain code paths |
| Numerics | numpy |
| Concurrency | concurrent.futures.ThreadPoolExecutor, subprocess.run (true parallel via separate interpreters), threading.Lock |
1.5 Data flow for the canonical "upload + render" workflow
1. POST /upload (multipart) ──► validate_master_filename()
► assign uuid4 job_id
► save to uploads/<job_id>/[MASTER]*.xlsx
► xlsx_to_nsm() → uploads/<job_id>/[MASTER]*.nsm
► touch_heartbeat(job_id)
◄ {job_id, filename}
2. POST /generate_step/<job_id>/init ──► run `show area`, `show device`,
`show l1_link`, `show attribute`
via run_ns_command()
► persist meta to .meta.json
◄ {areas, device_count, link_count,
attribute_titles}
3. GET /svg_grid_stream/<job_id> ──► fan out to ThreadPoolExecutor:
(Server-Sent Events) - export l1_diagram --type all_areas_tag
- export l1_diagram --type per_area_tag
- export l3_diagram --type all_areas
- export l3_diagram --type per_area
- export l2_diagram --area <each>
► each task → run_ns_command_subprocess()
(separate Python interpreter)
► moved files cached in _svg_mem_cache
◄ stream `data: {cell, files, svg_b64?}`
for each completed cell
◄ background AI-context generation
completes in parallel
◄ `data: {done: true, ai_filename}`
4. GET /download/<job_id>/<file> ──► serve from uploads/<job_id>/
5. (heartbeat_timeout seconds idle) ──► _cleanup_loop() rmtree(uploads/<job_id>/)
► drop _svg_mem_cache[job_id]
► drop _svg_grid_last_attr[job_id]
Note
The engine is deterministic. A given master file at a given commit always produces the same byte-for-byte output. This is intentional: diff-friendly diagrams let teams review topology changes the same way they review source code, and it's why subprocess timeouts default to a generous four hours rather than failing fast — large topologies are bounded by RAM and CPU, not by stochastic behavior.
2. System Requirements
2.1 Minimum
| Resource | Floor |
|---|---|
| CPU | 2 cores (any x86_64 / ARM64 server-class CPU) |
| RAM | 4 GB |
| Disk | 1 GB free (source + dependencies + room for one session of upload artifacts) |
| Python | 3.9 (CPython) |
| OS | Linux (any modern distro), macOS 12+, Windows 10/11/Server 2019+ |
| Network | TCP 5443 inbound (configurable) |
2.2 Recommended for shared / production use
| Resource | Recommendation |
|---|---|
| CPU | 4–8 cores. parallel_limit defaults to 6 and benefits from real cores up to that count |
| RAM | 8–16 GB. L3 all_areas rendering for very large fabrics is memory-intensive |
| Disk | 10 GB free. Per-session uploads/<job_id>/ directories can hold dozens of artifacts each |
| Python | 3.11+ for the speed-up over 3.9 in pandas / pyarrow paths |
| OS | Linux preferred (heartbeats and cleanup loop have been validated longest there) |
2.3 Python package floor
The exact pinned versions live in
network-sketcher_offline/requirements_offline.txt and
network-sketcher_online/requirements_online.txt. The functional floor:
| Package | Why |
|---|---|
flask |
Online server only |
openpyxl |
All editions (Excel I/O) |
pandas, pyarrow |
All editions (.nsm Parquet sections, I/O frames) |
python-pptx |
All editions (PowerPoint emitter) |
numpy |
All editions (layout math) |
networkx |
All editions (broadcast-domain detection, flow shortest-paths) |
cryptography |
Online edition (self-signed cert generation) |
tkinter, tkinterdnd2 |
Offline GUI only |
Important
The Online edition does not require Tk.
nsm_adapter.bootstrap()injects no-op stubs intosys.modules['tkinter'],sys.modules['tkinter.ttk'],sys.modules['tkinter.filedialog'], andsys.modules['tkinter.messagebox']before any engine module imports. This is how the Online edition runs cleanly in container images that have no display server.
3. Configuration Reference
This is the complete inventory of every configurable surface — JSON file, environment variable, and CLI flag. If a setting is not in one of these tables, it does not exist.
3.1 ns_web_config.json — the only config file
Located at network-sketcher_online/ns_web_config.json. Each top-level
key is {"value": ..., "description": "..."}; the loader strips the
description and keeps only the value. Keys whose names start with
underscore (_description) are ignored.
| Key | Type | Default | Description |
|---|---|---|---|
version |
string | 3.0.1h |
Display version; not used by code, kept for human inspection |
host |
string | localhost |
Bind address. 0.0.0.0 for all NICs, or pin a specific IP |
fqdn |
string | null | null |
Certificate Common Name. If null, falls back to host |
port |
int | 5443 |
HTTPS listen port |
max_file_size_mb |
int | 50 |
Upload size cap (Flask MAX_CONTENT_LENGTH); over-cap returns HTTP 413 |
subprocess_timeout |
int | null | 14400 |
Per-subprocess wall clock cap in seconds; null = no limit |
heartbeat_timeout |
int | 300 |
Seconds of inactivity before a session's uploads/<job_id>/ is wiped |
cleanup_interval |
int | 60 |
Seconds between idle-sweep runs in _cleanup_loop |
parallel_limit |
int | 6 |
max_workers for the SSE-driven SVG generator |
ssl_cert_path |
string | Certs/server.crt |
TLS cert path; relative to the config file or absolute |
ssl_key_path |
string | Certs/server.key |
TLS private key path |
ssl_cert_validity_days |
int | 3650 |
Validity window for auto-generated self-signed certs (10 years) |
ssl_key_size |
int | 4096 |
RSA key bits for auto-generated certs |
ssl_min_version |
string | TLSv1_2 |
One of TLSv1_2, TLSv1_3 |
ssl_max_version |
string | TLSv1_3 |
One of TLSv1_2, TLSv1_3 |
ssl_ciphers |
string | ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM |
OpenSSL cipher string |
log_level |
string | INFO |
One of DEBUG, INFO, WARNING, ERROR |
access_log_retention_days |
int | 90 |
Days of access_*.log to keep (purged on date roll-over and at startup) |
ai_context_button_label |
string | Copy & Open LLM |
UI button label (templated into HTML at request time) |
ai_context_button_url |
string | https://claude.ai |
URL the Copy & Open LLM button opens |
ai_context_show_commands |
array<string> | (15 show ... commands) |
Show-command list the AI Context exporter runs in parallel — the order is preserved in the output file. Default contains show area, show area_device, show area_location, show attribute, show device, show device_interface, show device_location, show l1_interface, show l1_link, show l2_broadcast_domain, show l2_interface, show l3_broadcast_domain, show l3_interface, show waypoint, show waypoint_interface |
Important
ssl_cert_validity_daysdefaults to 3650 (10 years). For production, replace the auto-generated cert with one from your internal CA — most browsers cap manually-trusted self-signed certs at 1 year of acceptance. See Section 7.4.
3.2 Environment variables
| Variable | Read in | Default | Effect |
|---|---|---|---|
NS_WEB_SERVER |
nsm_l2_diagram_create.py line 47 (Online and Offline copies) |
unset | When set to any non-empty string, engine code follows the headless / server code path. The Online server sets it explicitly in ns_web_start.py. |
NS_DEBUG_L3 |
nsm_l3_diagram_create.py line 324 (Online and Offline copies) |
0 |
Set to 1 to dump verbose L3-layout diagnostics to stdout. Useful for diagnosing L3 layout regressions; ship the captured stdout when filing issues. |
3.3 CLI flags (every ns_cli.py invocation)
3.3.1 Top-level verbs
| Verb | Required | Notes |
|---|---|---|
show <object> |
--master <path> |
Pure read. Output is a Python-array literal, suitable for ast.literal_eval |
add <object> ... |
--master <path> |
Mutates the master. The Online server adds --skip_sync for all but the last syncable command in a batch |
delete <object> ... |
--master <path> |
Mutates the master |
rename <object> ... |
--master <path> |
Mutates the master; preserves all back-pointers |
export <object> ... |
--master <path> |
Produces output artifacts (PPTX / SVG / Excel / TXT) |
3.3.2 export l1_diagram flags
| Flag | Values | Default | Effect |
|---|---|---|---|
--master |
path | (required) | Path to [MASTER]*.xlsx or [MASTER]*.nsm |
--type |
all_areas, all_areas_tag, per_area, per_area_tag |
all_areas_tag |
Single combined view vs. one diagram per area; _tag variants include interface name labels |
--attribute |
string | (none) | Title from the master file's Attribute sheet to color-code devices by |
--format |
pptx, svg |
pptx |
Output format |
3.3.3 export l2_diagram flags
| Flag | Values | Default | Effect |
|---|---|---|---|
--master |
path | (required) | Master file |
--area |
string | (required) | Area name (must exist in Master_Data_L2) |
--format |
pptx, svg |
pptx |
Output format |
3.3.4 export l3_diagram flags
| Flag | Values | Default | Effect |
|---|---|---|---|
--master |
path | (required) | Master file |
--type |
all_areas, per_area |
all_areas |
Single combined view vs. one diagram per area |
--attribute |
string | (none) | Color-coding attribute title |
--format |
pptx, svg |
pptx |
Output format |
3.3.5 Other export subcommands
| Subcommand | Flags | Effect |
|---|---|---|
device_file |
--master |
Emits [DEVICE]<basename>.xlsx with normalized L1/L2/L3/Attribute sheets |
ai_context_file |
--master, --accept-security-risk |
Emits [AI_Context]<basename>.txt. Requires either interactive YES confirmation or --accept-security-risk |
master_file_backup |
--master |
Timestamped backup of the master file |
master_file_nodata |
--master |
Empty-template master next to the input |
3.3.6 Mutating-command behavior flags
| Flag | Read by | Effect |
|---|---|---|
--skip_sync |
All add/delete mutating subcommands |
Skips the post-mutation L2/L3 sync. Used internally by the Online server to amortize sync cost across a batch — only the last syncable command in a run executes the sync |
--one_msg |
All show subcommands |
Concatenates output as a single string (rather than a list of lines). Used by the AI-context exporter to capture each show command's output in one piece |
3.4 First-run-state files
| File | Created by | Purpose |
|---|---|---|
ns_online.pid |
start_ns_online.py |
PID of the running server; consumed by stop_ns_online.py |
Certs/server.crt, Certs/server.key |
ns_web_start.py startup |
Self-signed TLS cert auto-generated if missing |
logs/access_YYYYMMDD.log |
_DailyFileHandler |
One file per local date; /heartbeat/* paths excluded |
logs/server.log |
start_ns_online.py |
Detached stdout/stderr |
uploads/<uuid4>/ |
/upload route |
Per-job working directory; auto-deleted on heartbeat expiry |
uploads/<uuid4>/.active_master |
set_active_master() |
Pointer to the currently-active master filename within the job dir (handles versioning) |
uploads/<uuid4>/.meta.json |
/generate_step/init |
Cached {areas, device_count, link_count, attribute_titles} for /restore |
4. Deployment & Scaling
Network Sketcher Online is a single-process Flask app — there is no
master/worker pool, no separate cache process, no message broker. This
keeps deployment simple but bounds horizontal scaling to running more
than one instance behind a load-balancer with sticky sessions (necessary
because per-session state lives in uploads/<job_id>/ on the local
filesystem).
4.1 Single-host deployment (typical)
# 1. Clone and install
git clone https://github.com/cisco-open/network-sketcher.git
cd network-sketcher
python -m pip install -r network-sketcher_online/requirements_online.txt
# 2. Optional: edit ns_web_config.json for your host/port
${EDITOR:-vi} network-sketcher_online/ns_web_config.json
# 3. Start
python start_ns_online.py
The server runs in the foreground of the Python process started by
Popen, but start_ns_online.py detaches it on Windows
(DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP) and creates a new POSIX
session on Linux/macOS (start_new_session=True).
4.2 Run as a systemd unit (Linux)
# /etc/systemd/system/network-sketcher.service
[Unit]
Description=Network Sketcher Online
After=network.target
[Service]
Type=forking
User=netsketcher
WorkingDirectory=/opt/network-sketcher
ExecStart=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/start_ns_online.py
ExecStop=/opt/network-sketcher/.venv/bin/python /opt/network-sketcher/stop_ns_online.py
PIDFile=/opt/network-sketcher/network-sketcher_online/ns_online.pid
Restart=on-failure
[Install]
WantedBy=multi-user.target
Tip
Use a dedicated unprivileged user for the service. The Online server never needs root; reading the config and binding port
5443work fine as a normal user (ports above 1024).
4.3 Containerized deployment
FROM python:3.11-slim
WORKDIR /opt/network-sketcher
COPY . .
RUN pip install --no-cache-dir \
-r network-sketcher_online/requirements_online.txt
# Persist Certs and uploads across container restarts
VOLUME ["/opt/network-sketcher/network-sketcher_online/Certs",
"/opt/network-sketcher/network-sketcher_online/uploads",
"/opt/network-sketcher/network-sketcher_online/logs"]
EXPOSE 5443
CMD ["python", "start_ns_online.py"]
4.4 Reverse proxy with TLS termination
If you front Network Sketcher with nginx/Caddy/Traefik that already holds your real cert:
- Set
host: 127.0.0.1and let the reverse proxy bind public ports. - Provide the upstream's own cert via
ssl_cert_path/ssl_key_path, or disable upstream TLS by replacing thessl.SSLContextinns_web_start.pywith a plain HTTP listener (currently not exposed as a config flag — patch required). - Forward
X-Forwarded-Forso access logs report client IPs.
Note
The default config bakes HSTS into every response with
max-age=31536000; includeSubDomains. Remove that header if you're intentionally serving HTTP behind a TLS-terminating proxy without HTTPS upstream.
4.5 Horizontal scaling
Each session's working files live on local disk under
uploads/<job_id>/. To run more than one Online instance:
- Place a load balancer in front (HAProxy / nginx / your cloud LB).
- Configure sticky sessions keyed on a cookie or header so a given
job_idalways lands on the same backend. - Keep
Certs/,uploads/, andlogs/on instance-local volumes — do not putuploads/on shared NFS unless you're willing to serialize all parallel writes.
4.6 Stopping cleanly
stop_ns_online.py finds every running ns_web_start.py process via
the OS process list (Windows: wmic process where ..., Linux/macOS:
pgrep -f ns_web_start.py) and falls back to the PID file as a
secondary source. It sends SIGTERM (POSIX) or taskkill /F /T /PID
(Windows), then waits up to 10 seconds for the process(es) to exit.
Warning
A SIGTERM during diagram generation aborts the in-flight subprocess. Sessions persist on disk and the user can re-issue the same generate request after the server is back up — but any partial output files in the job directory are not cleaned automatically (the next session's heartbeat-expiry sweep will eventually remove them).
5. Operations
5.1 Health check
There is no dedicated /health route. A useful liveness probe is the
root path:
curl -ksf https://127.0.0.1:5443/ -o /dev/null && echo OK
A readiness probe should additionally verify the cert paths exist and are readable.
5.2 Daily access log
logs/access_YYYYMMDD.log (one file per local date) contains one line
per request:
2026-04-28 14:07:31 198.51.100.42 POST /upload 200 upload="[MASTER]campus.xlsx"
2026-04-28 14:07:33 198.51.100.42 POST /generate_step/96d6b2a4-.../init 200
2026-04-28 14:07:35 198.51.100.42 GET /svg_grid_stream/96d6b2a4-... 200
2026-04-28 14:08:12 198.51.100.42 GET /download/96d6b2a4-.../[L1_DIAGRAM]AllAreasTag_campus.pptx 200 download="[L1_DIAGRAM]AllAreasTag_campus.pptx"
/heartbeat/<job_id> is suppressed from access logs (it would otherwise
fire every few seconds). Files older than access_log_retention_days
are deleted on startup and on each date roll-over.
Note
Network Sketcher does not rotate or compress access logs itself. If you ship them off-host, run
logrotate(or the equivalent) onlogs/access_*.log; the in-app cleanup is best-effort, not a backup.
5.3 Heartbeat eviction
Every successful upload registers a heartbeat with touch_heartbeat().
The browser front-end refreshes it every few seconds via
POST /heartbeat/<job_id>. The server's _cleanup_loop thread runs
every cleanup_interval seconds and rmtree-s any session whose
heartbeat is older than heartbeat_timeout. The same loop drops the
SVG memory cache and the per-session attribute cache.
5.4 Common operational checklists
After a clean install
- [ ]
python start_ns_online.pystarts without error - [ ]
https://localhost:5443/loads in a browser - [ ] Upload a master file, wait for the SSE grid to populate
- [ ] Confirm
logs/access_YYYYMMDD.logis being written - [ ] Confirm
Certs/server.crtwas created (4096-bit RSA, 10-year validity)
After a config change
- [ ]
python stop_ns_online.pyreportsStopped N process(es). - [ ]
python start_ns_online.pyreports a fresh PID - [ ] First request after restart returns 200
- [ ] Loaded
parallel_limit,port, etc. match what's now inns_web_config.json
Before a TLS cert renewal
- [ ] Backup
Certs/server.crtandCerts/server.key - [ ] Drop in the new cert and key with the same paths (or update
ssl_cert_path/ssl_key_path) - [ ] Restart
- [ ] Confirm the new fingerprint with
openssl x509 -in Certs/server.crt -noout -fingerprint -sha256
5.5 Backup strategy
| What | Why | How often |
|---|---|---|
network-sketcher_online/ns_web_config.json |
Config drift recovery | Whenever changed |
network-sketcher_online/Certs/ |
Avoid forced re-trust on rebuild | Whenever cert rotates |
| Source repo at the running commit | Reproducibility | At deploy time |
Master files ([MASTER]*.xlsx) and their backups |
Topology data is the only irreplaceable artifact | Daily, off-host |
uploads/<job_id>/ directories are not backup-worthy — they're
transient and auto-deleted.
5.6 Frequent cross-edition syncs
Both editions duplicate ~95% of the engine code (Offline ns_*.py and
Online ns_engine/nsm_*.py). When you backport a fix:
- Patch the Online file under
ns_engine/. - Diff against the Offline file under
network-sketcher_offline/— the names match one-to-one (ns_l2_diagram_create.py↔nsm_l2_diagram_create.py). - Apply the corresponding change to the Offline file.
- Run
python ns_cli.py show area --master "[MASTER]demo.xlsx"from each edition to confirm dispatcher integrity.
6. Performance
6.1 Where time goes
For a "first render after upload":
| Phase | Driver | Typical share |
|---|---|---|
xlsx_to_nsm conversion |
Master file size, sheet count | 5–15% |
show area/device/l1_link/attribute (init) |
Master file size | 5% |
L1 all_areas_tag diagram |
Device + link count, area count | 20–30% |
L1 per_area_tag (one per area) |
Per-area complexity | 10–20% |
L3 all_areas diagram |
Device + L3 interface count | 25–40% |
L3 per_area |
Per-area L3 density | 5–15% |
| L2 per area | L2 broadcast-domain density | 5–15% (× number of areas) |
| AI Context (parallel) | Master file size | runs concurrently with diagrams |
/svg_grid_stream parallelizes across parallel_limit workers; the
critical path is the slowest single subprocess. For most fabrics that's
the L3 all_areas render.
6.2 Tuning levers
| Lever | Effect | When to use |
|---|---|---|
Raise parallel_limit |
More concurrent diagrams | You have spare cores and RAM |
Lower parallel_limit |
Less RAM pressure | You're hitting OOMs / multiprocessing fallback errors |
Raise subprocess_timeout (or null) |
Bigger fabrics finish | You've sized RAM for the job and don't want timeouts |
Lower heartbeat_timeout |
Frees disk faster | Many short sessions, disk pressure |
Raise heartbeat_timeout |
Better UX for slow users | Few long sessions |
Use .nsm directly |
Skips xlsx→nsm conversion | Repeated renders against an unchanged master |
Use --type per_area |
Avoids the all_areas blow-up on huge fabrics |
Hundreds of devices, dozens of areas |
6.3 SVG caching path
After a successful subprocess render, the moved file's bytes are
cached in _svg_mem_cache[job_id][filename] under
_svg_mem_cache_lock. Subsequent requests to /svg_raw/<job_id>/...
serve from this cache, bypassing both the disk and the Windows AV-lock
re-read. The cache is dropped on heartbeat expiry.
6.4 Known performance hazards
Warning
On Windows, antivirus engines (notably Defender) re-scan files immediately after they're moved between directories. The engine mitigates this with
_wait_readable()(10-second wait with 150ms polling) before serving SVGs. If you see "File not readable after 10s" warnings inserver.log, your AV is holding files longer than 10 seconds — exclude theuploads/directory from real-time scanning or raise the timeout in_wait_readable.
7. Security
Network Sketcher's outputs are sensitive. A network topology
artifact reveals device names, vendor inventory (via interface naming),
IP plans, VRF / VPN structure, and broadcast-domain layout. Treat
every generated file as Internal at minimum, and the AI Context
file as Confidential.
7.1 What's in the box at first start
| Defense | Implementation | Notes |
|---|---|---|
| TLS by default | Auto-generated 4096-bit RSA self-signed cert with 10-year validity | Replace with a real cert before exposing beyond localhost |
| TLS version pinning | ssl_min_version: TLSv1_2, ssl_max_version: TLSv1_3 |
TLS 1.0/1.1 disabled by default |
| Cipher pinning | ssl_ciphers: ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM |
No RC4, no DES, no NULL |
Strict-Transport-Security |
max-age=31536000; includeSubDomains |
Forces browsers to HTTPS for one year |
X-Frame-Options |
DENY (default), SAMEORIGIN for /device_preview/<...> |
Prevents clickjacking; allows internal thumbnail iframes |
Content-Security-Policy |
default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; ...; frame-ancestors 'none' |
XSS / data-exfil reduction |
X-Content-Type-Options |
nosniff |
Stops MIME sniffing exploits |
Referrer-Policy |
strict-origin-when-cross-origin |
Limits referrer leakage |
| Job ID validation | UUID4 regex ^[a-f0-9\-]{36}$ |
Prevents path traversal via job id |
| Filename validation | Must start with [MASTER] and end in .xlsx or .nsm |
Rejects upload-traversal attempts |
| Upload size cap | max_file_size_mb (default 50 MB) |
Returns HTTP 413 over cap |
| Per-session isolation | UUID4 directories | One user can't read another's master file |
Caution
The default deployment has no authentication. Anyone who can reach the listening port can upload, render, and download. Never expose a default Network Sketcher Online directly to the internet. Put it behind an auth proxy (oauth2-proxy, Pomerium, your SSO IdP), a VPN, or a Tailscale tailnet before exposing it beyond localhost.
7.2 Sensitive data classification
| Artifact | Sensitivity | Why |
|---|---|---|
[MASTER]*.xlsx / *.nsm |
High | Contains the entire network plan |
[L1_DIAGRAM]* / [L2_DIAGRAM]* / [L3_DIAGRAM]* |
High | Visualizes device names, interface IDs, VLAN/VRF mappings |
[DEVICE]*.xlsx |
High | Tabular form of all interfaces and IPs |
[AI_Context]*.txt |
Highest | Plain-text dump intended for an external LLM |
logs/access_*.log |
Medium | Source IPs and request paths but no payloads |
logs/server.log |
Medium | May contain stack traces with file paths |
Warning
The AI Context file is the highest-risk output. It is designed to be pasted into a hosted LLM, which means it will be transmitted to a third party and may be retained for training (per the LLM vendor's TOS). The CLI requires either an interactive
YESconfirmation or--accept-security-risk; the web UI displays the same risk text in the tooltip. Audit any policy that allows unattended--accept-security-riskuse. Seensm_extensions.pyfor the export logic and the warning text.
7.3 Recommended access boundary
| Role | Should access | Should not access |
|---|---|---|
| Network engineer (read) | The web UI for their own organization's master files; downloaded diagrams under their own handling | Other tenants' master files; raw uploads/ directory on the host |
| Network engineer (write) | The web UI to mutate their own master via the CLI Commands panel | The host filesystem directly; other engineers' active sessions |
| Sysadmin / SRE | The host filesystem (/opt/network-sketcher), logs/, Certs/, ns_web_config.json |
Master file contents (can read them, but should treat as confidential by policy) |
| External / 3rd-party | Nothing (use a separate instance per tenant, or strong front-door auth) | All of the above |
7.4 Production TLS
The auto-generated cert is for development convenience. To replace it:
# Stop the server
python stop_ns_online.py
# Drop in CA-issued cert and key
cp /path/to/your/cert.pem network-sketcher_online/Certs/server.crt
cp /path/to/your/key.pem network-sketcher_online/Certs/server.key
# Or set custom paths in ns_web_config.json:
# "ssl_cert_path": {"value": "/etc/letsencrypt/live/sketcher.example.com/fullchain.pem"}
# "ssl_key_path": {"value": "/etc/letsencrypt/live/sketcher.example.com/privkey.pem"}
# "fqdn": {"value": "sketcher.example.com"}
python start_ns_online.py
Verify:
openssl s_client -connect sketcher.example.com:5443 -servername sketcher.example.com < /dev/null 2>&1 | openssl x509 -noout -subject -dates
7.5 Secrets handling
Network Sketcher itself stores no secrets in code or config. The only sensitive material on disk is:
- TLS private key (
Certs/server.key) — chmod0600, owned by the service user. - Master files in
uploads/<job_id>/— auto-cleaned on heartbeat expiry. For long-lived hosting, consider mountinguploads/on an encrypted volume.
There are no API keys, no database credentials, no third-party tokens.
The ai_context_button_url defaults to a public LLM URL but is purely
a hyperlink — Network Sketcher itself never calls out to it.
7.6 Sanitization recommendations before sharing diagrams
If you need to share a diagram externally (vendor RFP, conference talk, public docs):
- Make a copy of the master and run a
renamepass:rename device <real-name> <Device-N>for every devicerename area <real-area> <Area-N>for every area
- Run a
delete ip_addresspass on the L3 sheet (or set IPs toRFC1918-onlyrepresentatives viaadd ip_address). - Re-render the diagrams from the sanitized master.
- Do not redact PNG/PPTX exports manually — the underlying SVG
may still contain the un-redacted text in
<text>elements.
8. Developer Internals
8.1 Repository layout
network-sketcher/
├── README.md # Cisco Open project README
├── CONTRIBUTING.md
├── CODE_OF_CONDUCT.md
├── SECURITY.md
├── LICENSE # Apache 2.0
├── start_ns_online.py # Cross-platform daemonizer for Online
├── stop_ns_online.py # Cross-platform stop
├── network-sketcher_offline/ # Tkinter GUI + standalone CLI
│ ├── network_sketcher.py # GUI entrypoint
│ ├── ns_cli.py # Standalone CLI dispatcher
│ ├── ns_def.py # Shared helpers (file I/O, sheet→array, etc.)
│ ├── ns_dev.py # GUI internals
│ ├── ns_extensions.py # AI context, flow report, IP report, etc.
│ ├── ns_l1_*.py # L1 pipeline (5 files)
│ ├── ns_l2_*.py # L2 pipeline (3 files)
│ ├── ns_l3_*.py # L3 pipeline (3 files)
│ ├── ns_ddx_figure.py # PPTX renderer
│ ├── ns_egt_maker.py # Excel template maker
│ ├── ns_attribute_table_*.py
│ ├── ns_option_convert_to_master.py # External-system imports
│ ├── ns_sync_between_layers.py
│ ├── ns_vpn_diagram_create.py
│ ├── hook-pptx.py # PyInstaller hook
│ ├── ns_logo.png
│ ├── requirements_offline.txt
│ └── Sample_data/
└── network-sketcher_online/ # Flask server + headless engine
├── ns_web_start.py # Flask app: routes, security headers, parallelism
├── ns_web_config.json # The only config file
├── help_msg.json # Tooltip text for the web UI
├── requirements_online.txt
├── Certs/ # Auto-populated on first start
├── logs/ # Auto-populated on first request
├── static/ # JS deps (PptxViewJS, chart.umd, jszip, xlsx)
└── ns_engine/ # Headless engine (mirror of offline ns_*)
├── __init__.py
├── _ns_worker.py # Subprocess worker entry
├── nsm_adapter.py # Tk stubs + bootstrap + run_cli
├── nsm_io.py # xlsx ↔ nsm (Parquet) bridge
├── nsm_cli.py
├── nsm_def.py
├── nsm_l1_*.py / nsm_l2_*.py / nsm_l3_*.py
├── nsm_ddx_figure.py / nsm_ddx_svg*.py
├── nsm_extensions.py
├── nsm_svg_compute.py / nsm_svg_worker.py
└── nsm_sync_between_layers.py
8.2 Key invariants
Note
Master file integrity is preserved by design. The Online server never overwrites an uploaded master in place: any mutation creates a versioned copy (
[MASTER]<basename>_<n>.xlsx) and updates.active_master. This is the single most important behavior to preserve when patching.
Note
The engine is deterministic with respect to a given master + flag combination. If a refactor changes byte-for-byte output, that's a regression that needs review even if the diagrams look the same. Reviewers should diff the SVG output against a known-good master in
network-sketcher_offline/Sample_data/.
8.3 Module reference
ns_web_start.py — Flask app
Major helpers:
| Helper | Purpose |
|---|---|
_load_config() |
Loads ns_web_config.json, strips description, returns flat dict |
_load_help_messages() |
Loads help_msg.json for the UI tooltip system |
validate_master_filename(filename) |
Enforces [MASTER]*.xlsx/*.nsm |
sanitize_job_id(job_id) |
Enforces UUID4 regex |
find_master_file(work_dir) |
Picks the first [MASTER]* in a job dir |
get_active_master(work_dir) / set_active_master(work_dir, name) |
.active_master pointer |
run_ns_command(args) |
In-process CLI run via nsm_adapter.run_cli |
run_ns_command_isolated(args, work_dir, master_filename) |
Same, but in a temp subdir copy |
run_ns_command_subprocess(args, work_dir, master_filename) |
Separate Python interpreter (true parallelism) |
_wait_readable(path, timeout=10.0) |
Windows AV-lock backoff |
generate_ai_context_parallel(work_dir, master_filename) |
Fans out the configured show commands across a thread pool |
_DailyFileHandler |
Per-day log file with auto-cleanup |
_cleanup_loop() |
Heartbeat-driven session sweep |
_convert_svg_for_visio(svg_bytes) |
SVG2 → SVG1.1 transformations for Visio |
_convert_svg_to_drawio(svg_bytes) |
SVG → mxGraphModel XML |
nsm_adapter.py — engine bootstrap
| Component | Purpose |
|---|---|
_Noop |
Catch-all stub class returning self for any attribute / call. Backs every Tk widget stub |
Toplevel, Frame, Label, Entry, Text, StringVar, IntVar, BooleanVar, ... |
One stub class per Tk widget the engine references |
_TtkStub |
Equivalent stubs for tkinter.ttk |
_FileDialogStub |
askopenfilename, askopenfilenames, asksaveasfilename, askdirectory (all return empty) |
_MessageBoxStub |
showinfo, showwarning, showerror, askquestion, askokcancel, askyesno, askretrycancel |
bootstrap() |
Idempotent: injects all stubs into sys.modules, sets up sys.path, monkey-patches openpyxl.load_workbook to handle .nsm files |
_patch_openpyxl_for_nsm() |
Transparent .nsm → temp-.xlsx translation, cached for the session |
run_cli(args, cwd) |
In-process CLI runner with global _cli_lock (single command at a time per process) |
run_cli_isolated(args, work_dir, master_filename) |
Temp subdir + master copy + run + move outputs |
nsm_io.py — master file I/O
| Helper | Purpose |
|---|---|
SECTION_SHEET_MAP |
The 11 master sections with their (sheet_name, section_tag) |
_encode_cell(value) / _decode_cell(v) |
Type-preserving encoder using single-byte prefix (\x00I int, \x00F float, \x00B bool) |
_read_section_from_xlsx(path, ws_name, section_tag) |
Section reader that locates <<TAG>> and walks until the next tag or 3000 empty rows |
xlsx_to_nsm(xlsx_path, nsm_path) |
ZIP of Parquet files |
nsm_to_xlsx(nsm_path, xlsx_path) |
Reverse |
nsm_cli.py — verb dispatcher
The dispatcher reads argv_array and routes by literal verb match
('show' in argv_array, etc.), then by next_arg against the verb's
allow-list. The five subcommand allow-lists are documented in Section
3.3. Any unsupported verb or object prints the supported list and
exits.
Pipelines (nsm_l1_*.py, nsm_l2_*.py, nsm_l3_*.py)
Each layer has up to four files:
| Pattern | Role |
|---|---|
nsm_lN_master_create.py (L1 only) |
Builds the per-layer master from the high-level inputs |
nsm_lN_table_from_master.py |
Extracts the per-layer table view (shipped in [DEVICE]*.xlsx) |
nsm_lN_table_sync_master.py |
Keeps the per-layer table view consistent after master mutations |
nsm_lN_diagram_create.py |
Lays out the diagram and emits PPTX |
nsm_lN_svg_create.py |
Same layout, SVG emitter |
nsm_l1_diagram_sync_master.py (L1 only) |
Re-positions wires after device/area moves |
nsm_extensions.py — feature catalog
| Class | Purpose |
|---|---|
ai_context.export_ai_context_file(self, dummy) |
Writes [AI_Context]<basename>.txt from 15 show commands plus nsm_extensions_cmd_list.txt |
flow_report.create_device_flow_table(self, full_filepath_master, device_name) |
Per-device flow table with shortest-path routing (networkx) |
flow.add_routing_path_to_flow(...) / .get_shortest_path(...) / .append_flows_to_diagram(...) / .get_flow_item_list(...) / .export_flow_file(...) |
Flow-report internals |
ip_report.export_ip_report(self) / .get_folder_list(self) |
IP allocation report |
auto_ip_addressing.run_auto_ip(...) / .get_folder_list(...) / .get_auto_ip_param(...) |
Auto-assign IPs from a parameter set |
summary_diagram.export_summary_diagram(self) |
Single-slide topology summary |
cli_on_gui.execute_cli_commands(...) / .show_cli_input_window(...) / .show_command_output_window(...) |
Offline GUI CLI panel |
8.4 Code patterns and conventions
- Tk in the engine? All Tk imports go through stubs at module load
via
nsm_adapter.bootstrap(). The engine still callstk.Entry(),tk.StringVar()etc. — those resolve to the_Noop-backed classes. Don't try to remove the Tk imports; they're required for shared code with the Offline edition. - CLI verb parsing: simple membership checks (
'show' in argv_array) followed byget_next_arg(...)lookups. There's noargparseuse; if you add a flag, follow the existingif a == '--master': replace_next = Truepattern. - Versioning: never overwrite the input. Mutations write to
[MASTER]<basename>_<n>.xlsxand update.active_master. - Headless detection: check
os.environ.get('NS_WEB_SERVER')rather than detecting Tk presence — the stubs make Tk presence always-truthy in Online mode. - Subprocess vs. in-process: the Online server uses three
execution modes:
run_ns_command(in-process, holds_cli_lock),run_ns_command_isolated(in-process with master copy),run_ns_command_subprocess(separate interpreter for true parallelism). Pick the lowest-overhead one that satisfies your isolation needs.
8.5 Testing strategy
The repository ships no formal test suite. The maintainer-tested verification path is:
python ns_cli.py show area --master Sample_data/...— sanity check that the dispatcher and master loader are wired up.python ns_cli.py export l1_diagram --master Sample_data/... --type all_areas_tag— full L1 pipeline.python ns_cli.py export l2_diagram --master Sample_data/... --area <area>— full L2 pipeline per area.python ns_cli.py export l3_diagram --master Sample_data/... --type all_areas— full L3 pipeline.- End-to-end via the web UI for a final visual check.
When contributing, run all four CLI exports against
Sample_data/No_data_Master_file_with_AI_Context/[MASTER]no_data.xlsx
and visually diff the resulting .pptx against the previous version.
Tip
Sample data is intentionally minimal — it's a no-data master with the AI-context placeholder. For more thorough testing, build a small synthetic master with two areas, four devices, and a handful of L1 wires; commit it locally as a fixture.
8.6 Potential improvements
- Test suite. Even a smoke-test wrapping the four CLI exports would catch most regressions.
/healthendpoint. Currently health checks must hit/.- Auth integration hook. The server has no auth surface; a pluggable middleware that delegates to JWT / OIDC / mTLS would make multi-tenant deploys safer.
- Output retention policy.
uploads/<job_id>/currently lives for exactlyheartbeat_timeoutseconds; some users want "keep last 24 hours regardless of activity". Make the retention policy configurable separately from the heartbeat window. - Structured error response. Routes return mixed
{success, message}and{error}shapes. Standardizing would help programmatic clients. - Headless Tk removal. Long-term, the engine would be cleaner if
Tkreferences were stripped rather than stubbed. This is a large refactor (~25 modules) but enables packaging the engine as a library without the Adapter dance.
8.7 Contributing
The repository follows the Cisco Open contribution model:
- All code is Apache 2.0; every file carries the SPDX header
SPDX-License-Identifier: Apache-2.0andCopyright 2023 Cisco Systems, Inc. and its affiliates. - Open issues at github.com/cisco-open/network-sketcher/issues.
- See
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdat the repo root. - Security issues go to the address in
SECURITY.md, not the public issue tracker.
9. Integration Points
Network Sketcher is intentionally narrow at its edges. It does not talk to controllers, doesn't push configs, doesn't poll devices. Integration happens through file imports and exports.
9.1 Inbound: external-system imports (Offline GUI)
The Offline GUI's External Systems tab imports four formats into a
master file via ns_option_convert_to_master.py:
| Format | Source | Class in code |
|---|---|---|
| YAML | Cisco CML topology export | ns_option_convert_to_master_yaml |
| SVG | Visio export | ns_option_convert_to_master_svg |
| SVG | draw.io export | ns_option_convert_to_master_svg (same class, dialect-detected) |
| CSV | NetBox device export | ns_option_convert_to_master_csv |
Two helper classes round out the import flow:
ns_overwrite_line_to_master_yaml— overlays a YAML topology onto an existing master.ns_l3_config_to_master_yaml— adds L3 details from a YAML config bundle.
9.2 Outbound: derived export formats
| Format | Trigger | Implementation |
|---|---|---|
.pptx |
export l1_diagram / l2_diagram / l3_diagram (default --format pptx) |
nsm_ddx_figure.py (python-pptx) |
.svg |
--format svg on diagram exports |
nsm_l1_svg_create.py / nsm_l2_svg_create.py / nsm_l3_svg_create.py |
Visio-friendly .svg |
GET /download_visio/<job_id>/<file> |
_convert_svg_for_visio() in ns_web_start.py |
draw.io .drawio (XML) |
GET /download_drawio/<job_id>/<file> |
_convert_svg_to_drawio() in ns_web_start.py |
[DEVICE]<basename>.xlsx |
export device_file |
Composed from L1/L2/L3/Attribute pipelines |
[AI_Context]<basename>.txt |
export ai_context_file |
nsm_extensions.ai_context.export_ai_context_file() |
.nsm (ZIP of Parquet) |
Auto on Online upload | nsm_io.xlsx_to_nsm() |
| ZIP of all session artifacts | GET /download_all/<job_id> |
Built in-memory by download_all() |
9.3 LLM integration
The AI Context flow is the only outbound call path:
- The web UI (or
nsm_extensions.ai_context.export_ai_context_file) builds a deterministic text bundle from the configured show commands plusnsm_extensions_cmd_list.txt. - The user clicks Copy & Open LLM (or hits the Download
button). The button label and target URL are configured via
ai_context_button_labelandai_context_button_url(defaulthttps://claude.ai). - The clipboard content is the AI Context plus the user's natural- language prompt.
- The user pastes into the LLM, gets back CLI commands, pastes them into the CLI Commands box, and hits Run.
Network Sketcher itself never sends the data anywhere — the user controls the transmission.
9.4 Programmatic engine usage
You can embed the engine in another Python program by bootstrapping
once and calling run_cli() repeatedly:
from ns_engine.nsm_adapter import bootstrap, run_cli
bootstrap()
result = run_cli([
'export', 'l1_diagram',
'--master', '/path/to/[MASTER]campus.xlsx',
'--type', 'all_areas_tag',
'--format', 'svg',
])
print(result.returncode) # 0 on success
print(result.stdout) # captured stdout
print(result.stderr) # captured stderr
run_cli holds a single global _cli_lock, so calls are serialized
within one process. For true parallelism, follow the pattern in
run_ns_command_subprocess and shell out to _ns_worker.py instead.
Tip
If you're embedding the engine, prefer
run_cli_isolated()overrun_cli()for any mutating command — it copies the master into a temp directory first, so a crash in your code can't corrupt the caller's master file.
9.5 Front-end JS dependencies
The Online edition's UI uses these vendored JS libraries (under
network-sketcher_online/static/):
| File | Purpose |
|---|---|
PptxViewJS.min.js |
In-browser PowerPoint preview (zoom, pan) |
chart.umd.min.js |
Chart rendering for analytics panels |
jszip.min.js |
Client-side ZIP support for previews |
xlsx.full.min.js |
In-browser Excel preview |
logo.png |
Brand asset |
These are vendored to keep the deployment self-contained — no CDN calls happen at runtime. If you upgrade a library, replace the file in place and restart the server.