pancloud-nodejs User Guide

pancloud-nodejs is the Palo Alto Networks Application Framework SDK for Node.js. It is a TypeScript library that wraps the three Cortex Data Lake services that PAN exposes to third-party tooling — Logging Service, Event Service, and Directory Sync Service — and adds the OAuth2 plumbing, automatic access-token refresh, and retry logic you would otherwise have to write yourself.

This guide is for the developer building a Node.js application that needs to read PAN-OS firewall logs, subscribe to log events, or query Active Directory data out of a Cortex Data Lake instance. It is organized as a linear path from the first npm install through credential setup, day-to-day querying patterns, and the failure modes you will hit in production.

1. Quick Start

The fastest way to talk to a Cortex Data Lake from Node.js: install the package, set three environment variables, and call autoCredentials(). The SDK figures out the credential type from the variables you set, hands you back a refresh-aware Credentials object, and you are ready to query.

Install

npm install pancloud-nodejs

Tip

The package is published as pancloud on its package.json name field but is consumed under the GitHub repository name. The TypeScript test suite imports it as from 'pancloud-nodejs' and that is the import path shown throughout this guide.

A 5-line query

Save this as quickstart.ts:

import { autoCredentials, LoggingService } from 'pancloud-nodejs'

async function main(): Promise<void> {
    const cred = await autoCredentials()
    const ls = LoggingService.factory(cred)
    const job = await ls.query({
        query: "SELECT * FROM panw.traffic LIMIT 10",
        startTime: Math.floor(Date.now() / 1000) - 3600,
        endTime: Math.floor(Date.now() / 1000)
    })
    console.log(`Query ${job.queryId}: ${job.queryStatus}`)
}

main().catch(e => console.error(e))

Provide credentials via environment variables (the simplest path):

export PAN_CLIENT_ID="<your-oauth2-client-id>"
export PAN_CLIENT_SECRET="<your-oauth2-client-secret>"
export PAN_REFRESH_TOKEN="<your-refresh-token>"
export PAN_ENTRYPOINT="https://api.us.paloaltonetworks.com"
node --loader ts-node/esm quickstart.ts

Expected output:

PANCLOUD: {"source":"AutoCredentials","message":"Using memory based credentials provider"}
PANCLOUD: {"source":"defaultCredentialsFactory","message":"Got 'client_id'"}
PANCLOUD: {"source":"defaultCredentialsFactory","message":"Got 'client_secret'"}
PANCLOUD: {"source":"LoggingService","message":"Creating new LoggingService object for entryPoint https://api.us.paloaltonetworks.com"}
Query 1234abcd-...: RUNNING

Note

The query returns a JobResult that may be in RUNNING, FINISHED, JOB_FINISHED, JOB_FAILED, or CANCELLED state. For the synchronous result, see LoggingServiceClient (the streaming wrapper) or the auto-poll callBack pattern shown later in this guide.

What you get

• OAuth2 management — the SDK refreshes the access token automatically when it is within 5 minutes of expiry (accTokenGuardTime default).
• Three service clients — LoggingService, EventService, DirectorySyncService, all reached via a factory(cred) static method.
• Retry-with-backoff — every HTTP call is wrapped in a retrier (default 3 attempts, 100 ms apart).
• Async log streaming — LoggingServiceClient is a Node Readable stream that yields log records as they become available.
• Multi-datalake / multi-tenant — CortexHubHelper and CortexCredentialProvider cover the OAuth2 authorization-code flow plus per-tenant credential storage.

What to try next

• Section 2 walks through the supported install methods.
• Section 3 covers the four credential strategies the SDK ships with.
• Section 4 has the full surface area — every public class and its main methods.
• Section 5 has copy-paste recipes for the eight workflows most apps need.

2. Installation

Requirements

Important

You need OAuth2 credentials issued by Palo Alto Networks before any of this works. That means a registered application in the Cortex hub plus a client_id, a client_secret, and either a refresh_token (for long-lived headless usage) or a developer token (for quick testing). The SDK does not issue credentials; it only consumes them.

Install methods

npm

npm install pancloud-nodejs

yarn

yarn add pancloud-nodejs

pnpm

pnpm add pancloud-nodejs

From source

git clone https://github.com/PaloAltoNetworks/pancloud-nodejs.git
cd pancloud-nodejs
npm run build:library

The build:library script empties dist/, runs tsc -p tsconfig.json, and emits CommonJS modules with declaration files alongside.

Verify the install

Once installed, this one-liner confirms the import path works:

node -e "console.log(Object.keys(require('pancloud-nodejs')).slice(0, 6))"

Expected output:

[
  'defaultCredentialsFactory',
  'DevTokenCredentials',
  'autoCredentials',
  'LoggingService',
  'LoggingServiceClient',
  'EventService'
]

Note

The package's package.json declares "main": "dist/index.js" and has an empty dependencies block — every external API is satisfied by the Node standard library. Your bundle gets no transitive runtime dependencies from this SDK.

TypeScript projects

For TypeScript consumers, no @types/* package is required — type declarations ship in the package. The tsconfig.json recommended for an SDK consumer matches the test suite shipped with this repo:

{
    "compilerOptions": {
        "noImplicitAny": true,
        "strictNullChecks": true,
        "target": "es2017",
        "module": "commonjs"
    }
}

Warning

The Symbol.iterator pattern that EventService exposes (for (let p of es) { ... }) requires an iteration target of es2015 or later. If you set target: "es5" you must also enable "downlevelIteration": true to get the same behaviour.

3. Configuration

The only thing the SDK has to be told is how to get an access token. Everything else (retry counts, timeouts, log level, channel ID) has a sensible default.

The four credential strategies

The SDK ships four ways to produce a Credentials object. Pick the one that matches your deployment shape:

Environment variables

autoCredentials() and the two factory functions read from process.env. The names are fixed:

Important

Treat PAN_CLIENT_SECRET and PAN_REFRESH_TOKEN as long-lived secrets. A leaked refresh token means full access to your Cortex Data Lake until someone manually revokes it. Store them in a vault, inject at runtime, never commit to source control.

The two regional entry points

The SDK exports a string-literal type EntryPoint with exactly two allowed values:

export type EntryPoint =
    'https://api.eu.paloaltonetworks.com' |
    'https://api.us.paloaltonetworks.com'

A convenience map region2EntryPoint translates 'americas' and 'europe' to the corresponding URLs.

Tip

If you set PAN_ENTRYPOINT to anything else, the SDK still accepts it (it casts string to EntryPoint) but you have left the supported matrix. Stick to one of the two literal values unless PAN issues a new regional API endpoint.

Common knobs (all optional)

Both EventService.factory() and LoggingService.factory() accept the same EmitterOptions configuration object, which extends CoreOptions:

For everything else, see the Configuration Reference in the Admin Guide.

4. Core Usage

This section walks through the four pieces every application touches: credentials, the three service clients, and the iteration patterns the clients support.

4.1 Credentials

Every service factory takes a Credentials instance. Credentials is abstract — the SDK ships four concrete subclasses plus a factory for each.

autoCredentials() — let the SDK pick

The simplest entry point. It inspects environment variables and chooses:

1. If PAN_CLIENT_ID + PAN_CLIENT_SECRET + PAN_REFRESH_TOKEN are all set, it uses the memory provider (refresh-token flow).
2. Else if PAN_ACCESS_TOKEN is set, it builds a static credentials object (no refresh).
3. Else it falls back to developer token mode and reads PAN_DEVELOPER_TOKEN.

import { autoCredentials } from 'pancloud-nodejs'

const cred = await autoCredentials()
// cred is now a Credentials instance ready to hand to any factory

You can override individual fields without setting environment variables:

const cred = await autoCredentials({
    accessToken: '<jwt>',
    entryPoint: 'https://api.eu.paloaltonetworks.com'
})

defaultCredentialsProviderFactory() — explicit memory provider

When you want the refresh-token flow but do not want it picked from env:

import { defaultCredentialsProviderFactory } from 'pancloud-nodejs'

const cred = await defaultCredentialsProviderFactory({
    clientId:     '<oauth2-client-id>',
    clientSecret: '<oauth2-client-secret>',
    refreshToken: '<oauth2-refresh-token>',
    entryPoint:   'https://api.us.paloaltonetworks.com'
})

Internally this returns a Credentials bound to a one-datalake DefaultCredentialsProvider instance with the fixed datalake ID 'DEFAULT'.

fsCredentialsFactory() — encrypted file cache

For headless services that want refresh tokens persisted across restarts. Tokens are stored in ${PAN_PREFIX}_PANCLOUD_CONFIG.json (default PAN_PANCLOUD_CONFIG.json) encrypted with aes128 derived from the secret you provide:

import { fsCredentialsFactory } from 'pancloud-nodejs'

const provider = await fsCredentialsFactory({
    clientId:     process.env.PAN_CLIENT_ID!,
    clientSecret: process.env.PAN_CLIENT_SECRET!,
    secret:       process.env.PAN_FS_SECRET!  // encryption key (any string)
})

const cred = await provider.registerManualDatalake(
    'datalake-1',
    'https://api.us.paloaltonetworks.com',
    '<refresh-token>'
)

DevTokenCredentials — quick demos

Useful with the apiexplorer.rocks flow if you have a developer token:

import { DevTokenCredentials } from 'pancloud-nodejs'

const cred = new DevTokenCredentials({
    developerToken: process.env.PAN_DEVELOPER_TOKEN,
    entryPoint:     'https://api.us.paloaltonetworks.com'
})
await cred.retrieveAccessToken()

defaultCredentialsFactory() — static, no refresh

For tests where you already have a fresh JWT in hand:

import { defaultCredentialsFactory } from 'pancloud-nodejs'

const cred = defaultCredentialsFactory(
    'https://api.us.paloaltonetworks.com',
    '<eyJ...JWT...>'
)

Warning

defaultCredentialsFactory() returns a StaticCredentials instance. Its retrieveAccessToken() is a no-op — when the JWT expires, queries will start failing with 401 errors. Use it for fixtures only.

Automatic refresh

Every concrete Credentials subclass implements autoRefresh(). The service clients call it before every HTTP request via CoreClass.checkAutoRefresh(). If the access token is within guardTime (5 minutes by default) of expiry, the SDK calls retrieveAccessToken() and updates the Authorization header transparently.

// You almost never call these directly; the service clients do.
await cred.getAccessToken()      // Returns the current access token
await cred.getExpiration()       // Returns the unix epoch validity
await cred.autoRefresh()         // Refreshes if needed, returns validUntil
cred.getEntryPoint()             // Returns the regional entry point

4.2 LoggingService — the main query API

LoggingService runs SQL-style queries against the Cortex Data Lake's log indexes (panw.traffic, panw.threat, panw.system, etc.) and returns results in batches.

Synchronous query (poll-yourself)

import { autoCredentials, LoggingService } from 'pancloud-nodejs'

const cred = await autoCredentials()
const ls = LoggingService.factory(cred)

const job = await ls.query({
    query:     "SELECT * FROM panw.threat WHERE risk-of-app > 3",
    startTime: 1577836800,
    endTime:   1580515200,
    maxWaitTime: 30000
})

console.log(`queryId: ${job.queryId}, status: ${job.queryStatus}`)

// If the result didn't arrive on the first call, poll until done:
let next = job
while (next.queryStatus !== 'JOB_FINISHED' && next.queryStatus !== 'JOB_FAILED') {
    next = await ls.poll(next.queryId, next.sequenceNo + 1, 30000)
}

Asynchronous query with callback (the recommended pattern)

When you provide a callBack.event handler, the SDK starts an auto-poller in the background. Every time a sequence of results is ready, your handler is called with an EmitterInterface<any[]> containing the records.

import { autoCredentials, LoggingService, EmitterInterface } from 'pancloud-nodejs'

const cred = await autoCredentials()
const ls = LoggingService.factory(cred)

const completed = await ls.query({
    query:     "SELECT * FROM panw.traffic",
    startTime: Math.floor(Date.now() / 1000) - 3600,
    endTime:   Math.floor(Date.now() / 1000),
    logType:   'panw.traffic',
    callBack: {
        event: (e: EmitterInterface<any[]>) => {
            if (e.message) {
                console.log(`Got ${e.message.length} ${e.logType} records`)
            }
        }
    }
})
console.log(`Job ${completed.queryId} done, ${ls.getLsStats().records} records total`)

Streaming query with LoggingServiceClient

LoggingServiceClient extends Node's Readable stream interface. It lazy-starts the query on the first _read() and pushes each batch as a single chunk in object mode:

import { autoCredentials, LoggingService, LoggingServiceClient } from 'pancloud-nodejs'

const cred = await autoCredentials()
const ls = LoggingService.factory(cred)

const stream = new LoggingServiceClient(ls, {
    query:     "SELECT * FROM panw.traffic",
    startTime: Math.floor(Date.now() / 1000) - 3600,
    endTime:   Math.floor(Date.now() / 1000),
    retries:   20,        // attempts before giving up on RUNNING state
    delay:     200        // ms between RUNNING-state polls
})

stream.on('data', (records: any[]) => {
    console.log(`stream chunk: ${records.length} records`)
})
stream.on('end', () => console.log('stream finished'))
stream.on('error', e => console.error('stream error:', e))

Job statuses

Every result carries a queryStatus field:

Other methods

4.3 EventService — the streaming subscription API

Where LoggingService runs ad-hoc SQL, EventService opens a long-lived channel (default name EventFilter) into which the Application Framework publishes matching events as they happen. Your code applies a filter to that channel — a list of tables and optional WHERE clauses — and then polls for events.

Set a filter and poll synchronously

import { autoCredentials, EventService, EsFilterBuilderCfg } from 'pancloud-nodejs'

const builderCfg: EsFilterBuilderCfg = {
    filter: [
        { table: "panw.traffic", timeout: 1000, batchSize: 8000 },
        { table: "panw.dpi",     timeout: 1000, batchSize: 8000 },
        { table: "panw.threat",  where: 'where risk-of-app > 3' }
    ],
    flush: false,
    filterOptions: {}
}

const cred = await autoCredentials()
const es = EventService.factory(cred)
await es.filterBuilder(builderCfg)

const events = await es.poll()
events.forEach(e => {
    console.log(`Event Type: ${e.logType}, Record Count: ${e.event.length}`)
})

Set a filter and stream asynchronously

When you provide a callBack in filterOptions, the SDK auto-polls and pushes each event batch through the handler:

const builderCfg: EsFilterBuilderCfg = {
    filter: [
        { table: "panw.traffic", timeout: 1000 },
        { table: "panw.threat",  where: 'where risk-of-app > 3' }
    ],
    filterOptions: {
        callBack: {
            event: e => {
                if (e.message) console.log(`got ${e.message.length} ${e.logType} events`)
            }
        },
        poolOptions: { ack: true, pollTimeout: 1000 }
    }
}

const es = EventService.factory(cred)
await es.filterBuilder(builderCfg)
// Auto-poll starts in the background; let it run for 60 seconds:
await new Promise<void>(resolve => setTimeout(resolve, 60000))
es.pause()
await es.clearFilter(true)
console.log(JSON.stringify(es.getEsStats(), undefined, ' '))

Iterating without a callback

EventService implements Iterable<Promise<EsEvent[]>>. You can drive the polling loop directly with for...of:

const es = EventService.factory(cred)
await es.filterBuilder(builderCfg)
let iterations = 10
for (const promise of es) {
    if (iterations-- === 0) break
    const response = await promise
    response.forEach(e => console.log(`${e.event.length} ${e.logType} events`))
}
await es.clearFilter()

Channel control methods

Warning

The Event Service rejects the tms.analytics, tms.config, tms.system, and tms.threat log types — only tms.traps is allowed from the tms.* namespace. filterBuilder() validates this before the HTTP call and throws PanCloudError(CONFIG) if you pass an invalid table.

4.4 DirectorySyncService — Active Directory data

DirectorySyncService exposes the directory inventory (computers, containers, groups, organizational units, users) that the PAN-OS-integrated Directory Sync agent has uploaded.

import { autoCredentials, DirectorySyncService } from 'pancloud-nodejs'

const cred = await autoCredentials()
const dss = await DirectorySyncService.factory(cred)

const domains = await dss.domains()
console.log(`${domains.length} domains synced`)

const userCount = await dss.count(domains[0].dns, 'users')
console.log(`${userCount} users in ${domains[0].dns}`)

const userPage = await dss.query('users', {
    domain: domains[0].dns,
    pageNumber: 1,
    pageSize: 100
})
console.log(`page 1 has ${userPage.result.length} entries`)

The five object classes are exposed as the type DssObjClass:

type DssObjClass = "containers" | "computers" | "ous" | "groups" | "users"

DssQueryFilter supports name-based matching with operators startWith, contain, equal against attributes Common-Name, Distinguished Name, Name, SAM Account Name, SID, User Principal Name, plus pagination via pageNumber and pageSize (max 1000 per page).

4.5 Logging the SDK itself

Every operation logs through a single commonLogger instance:

import { setLogLevel, setLogger, LogLevel } from 'pancloud-nodejs'

setLogLevel(LogLevel.DEBUG)   // Print everything

// Or replace the logger entirely:
setLogger({
    level: LogLevel.INFO,
    error: (e) => myMonitoring.report(e),
    alert: (src, msg, name) => console.warn(`[${src.className}] ${msg}`),
    info:  (src, msg, name) => {},
    debug: (src, msg, name, payload) => {}
})

LogLevel values: DEBUG = 0, INFO = 1, ALERT = 2, ERROR = 3. The default is INFO.

Tip

Output lines are prefixed with PANCLOUD: and contain a JSON object of {source, message, ...}. Pipe stdout through jq or your log aggregator's JSON parser to slice them up.

5. Common Tasks

This section is a copy-paste cookbook for the eight workflows that show up in nearly every integration.

5.1 Run a one-shot logs query and print the records

Goal: retrieve traffic logs from the last hour, log how many you got, exit.

Prerequisites: PAN_CLIENT_ID, PAN_CLIENT_SECRET, PAN_REFRESH_TOKEN, PAN_ENTRYPOINT set in environment.

Steps:

import { autoCredentials, LoggingService, EmitterInterface } from 'pancloud-nodejs'

async function main(): Promise<void> {
    const cred = await autoCredentials()
    const ls = LoggingService.factory(cred)

    let totalRecords = 0
    await ls.query({
        query:     "SELECT * FROM panw.traffic LIMIT 1000",
        startTime: Math.floor(Date.now() / 1000) - 3600,
        endTime:   Math.floor(Date.now() / 1000),
        logType:   'panw.traffic',
        callBack: {
            event: (e: EmitterInterface<any[]>) => {
                if (e.message) {
                    totalRecords += e.message.length
                    console.log(`batch of ${e.message.length} (running total ${totalRecords})`)
                }
            }
        }
    })
    console.log(`done, ${totalRecords} records`)
}

main().catch(e => console.error(e))

Verify: the script prints one or more batch of N lines, then done, N records. The LoggingService stats reflect the same totals via ls.getLsStats().

If it fails: start by raising the log level (setLogLevel(LogLevel.DEBUG)). A 401 means the refresh token is bad — re-issue from the Cortex hub. A PARSER error means the endpoint returned non-JSON, usually a transient upstream issue; the SDK already retried 3 times.

5.2 Stream logs to a file as a Node Readable

Goal: pipe a query result straight to disk without buffering it in memory.

Steps:

import { createWriteStream } from 'fs'
import { autoCredentials, LoggingService, LoggingServiceClient } from 'pancloud-nodejs'

const cred = await autoCredentials()
const ls = LoggingService.factory(cred)

const stream = new LoggingServiceClient(ls, {
    query:     "SELECT * FROM panw.threat",
    startTime: Math.floor(Date.now() / 1000) - 86400,
    endTime:   Math.floor(Date.now() / 1000)
})

const out = createWriteStream('threats.ndjson', { encoding: 'utf8' })
stream.on('data', (records: any[]) => {
    for (const r of records) out.write(JSON.stringify(r) + '\n')
})
stream.on('end', () => out.end())
stream.on('error', e => { out.end(); console.error(e) })

Verify: wc -l threats.ndjson matches ls.getLsStats().records.

If it fails: LoggingServiceClient throws if the query stays in RUNNING for more than retries polls (default 20, 200 ms apart = 4 seconds). Pass a higher retries for very large queries.

5.3 Subscribe to live traffic events and ack each batch

Goal: keep a long-running process consuming panw.traffic events, acknowledging each batch so the channel does not back up.

Steps:

import { autoCredentials, EventService, EsFilterBuilderCfg } from 'pancloud-nodejs'

const cred = await autoCredentials()
const es = EventService.factory(cred)

const cfg: EsFilterBuilderCfg = {
    filter: [{ table: "panw.traffic", timeout: 1000, batchSize: 8000 }],
    filterOptions: {
        callBack: {
            event: e => console.log(`${e.message?.length ?? 0} events at ${new Date().toISOString()}`)
        },
        poolOptions: { ack: true, pollTimeout: 1000 }
    }
}
await es.filterBuilder(cfg)

process.on('SIGINT', async () => {
    es.pause()
    await es.clearFilter(true)
    process.exit(0)
})

Verify: the process prints a line every poll iteration. es.getEsStats() shows polls climbing and acks matching it.

If it fails: if you see no output, the filter probably has no matching traffic. Drop the WHERE clause and try panw.system, which is always populated.

5.4 Page through Active Directory users

Goal: enumerate every user in a synced domain, 500 at a time.

Steps:

import { autoCredentials, DirectorySyncService } from 'pancloud-nodejs'

const cred = await autoCredentials()
const dss = await DirectorySyncService.factory(cred)
const domains = await dss.domains()
const dom = domains[0].dns

const total = await dss.count(dom, 'users')
console.log(`paging ${total} users in ${dom}`)

let page = 1
while (true) {
    const resp = await dss.query('users', { domain: dom, pageNumber: page, pageSize: 500 })
    if (resp.result.length === 0) break
    console.log(`page ${page}: ${resp.result.length} entries (count=${resp.count})`)
    if (resp.unreadResults === undefined || resp.unreadResults === 0) break
    page++
}

Verify: the sum of result.length across pages equals the original count.

If it fails: a 4xx response often means the directory hasn't been synced yet — dss.domains()[0].status.statusUpdateTime will be 0.

5.5 Spin the SDK up for a quick test with a static token

Goal: validate connectivity in a 30-second test without provisioning a refresh-token flow.

Steps:

import { defaultCredentialsFactory, LoggingService } from 'pancloud-nodejs'

const cred = defaultCredentialsFactory(
    'https://api.us.paloaltonetworks.com',
    process.env.PAN_ACCESS_TOKEN!
)

const ls = LoggingService.factory(cred)
const job = await ls.query({
    query:     "SELECT * FROM panw.traffic LIMIT 1",
    startTime: Math.floor(Date.now() / 1000) - 600,
    endTime:   Math.floor(Date.now() / 1000)
})
console.log(`status: ${job.queryStatus}, queryId: ${job.queryId}`)

Verify: queryStatus is one of RUNNING, FINISHED, JOB_FINISHED.

If it fails: if the SDK throws CONFIG: not a valid JWT access token, the value in PAN_ACCESS_TOKEN is missing the three dot-separated segments. Re-fetch a fresh token.

5.6 Enable the L2/L3 correlation engine

Goal: receive synthesized correlation events that join L2 (DPI / MAC) and L3 (traffic / threat) flows by sessionid.

Prerequisites: filter must include both DPI events for L2 and panw.traffic or panw.threat for L3.

Steps:

import { autoCredentials, EventService, EsFilterBuilderCfg, L2correlation, EmitterInterface } from 'pancloud-nodejs'

const cred = await autoCredentials()
const es = EventService.factory(cred, {
    l2Corr: { timeWindow: 120, absoluteTime: false, gcMultiplier: 0 }
})

const cfg: EsFilterBuilderCfg = {
    filter: [
        { table: "panw.traffic", timeout: 1000 },
        { table: "panw.dpi",     timeout: 1000 }
    ],
    filterOptions: {
        callBack: {
            corr: (e: EmitterInterface<L2correlation[]>) => {
                if (e.message) {
                    console.log(`${e.message.length} correlation events`)
                    e.message.forEach(x => console.log(`  ${x.src} <-> ${x.dst} mac=${x["extended-traffic-log-mac"]}`))
                }
            }
        },
        poolOptions: { ack: true, pollTimeout: 1000 }
    }
}
await es.filterBuilder(cfg)

setTimeout(async () => {
    es.pause()
    await es.clearFilter(true)
    console.log(JSON.stringify(es.getEsStats(), undefined, ' '))
}, 120000)

Verify: es.getEsStats().correlationStats shows dbInserts > 0 and dbWaterMark > 0.

If it fails: without panw.dpi events the engine has no L2 data and will never produce correlations. Make sure the firewall sending data is configured to forward extended-application logs.

5.7 Decode PCAP payloads from threat events

Goal: save every PCAP payload embedded in a panw.threat event to a separate .pcap file.

Steps:

import { writeFileSync } from 'fs'
import { autoCredentials, EventService, EsFilterBuilderCfg, EmitterInterface } from 'pancloud-nodejs'

const cred = await autoCredentials()
const es = EventService.factory(cred)

let n = 0
const cfg: EsFilterBuilderCfg = {
    filter: [{ table: "panw.threat", timeout: 1000 }],
    filterOptions: {
        callBack: {
            pcap: (p: EmitterInterface<Buffer>) => {
                if (p.message) {
                    const name = `pcap${('00' + n++).slice(-3)}.pcap`
                    writeFileSync(name, p.message)
                    console.log(`wrote ${name} (${p.message.length} bytes)`)
                }
            }
        },
        poolOptions: { ack: true, pollTimeout: 1000 }
    },
    flush: true
}
await es.filterBuilder(cfg)

Verify: files appear on disk; each is a valid libpcap file.

If it fails: Util.pcaptize() returns null if the event has no pcap property, so no file is written. Switch to a higher-volume table like panw.dpi to confirm the pipeline before debugging the threat filter.

5.8 Turn on verbose tracing

Goal: see every HTTP request the SDK makes, with the JSON payload.

Steps:

import { setLogLevel, LogLevel, autoCredentials, EventService } from 'pancloud-nodejs'

setLogLevel(LogLevel.DEBUG)

const cred = await autoCredentials()
const es = EventService.factory(cred)
const f = await es.getFilters()
console.log(JSON.stringify(f, undefined, ' '))

Verify: stdout now shows PANCLOUD: ... debug lines for header updates, fetches, and JSON payloads (truncated to 300 bytes).

If it fails: if you replaced the logger via setLogger(), your custom debug implementation may be swallowing messages. Confirm by reverting to the default commonLogger first.

6. Troubleshooting

Diagnostic decision tree

1. Does autoCredentials() succeed? If not, check env vars (PAN_CLIENT_ID etc.) and PAN_ENTRYPOINT.
2. Does the first HTTP call succeed? If you get IDENTITY or 401, the refresh token is invalid or the entry point is wrong.
3. Is the query returning records? Confirm with setLogLevel(LogLevel.DEBUG) and look at the JSON payload echoed in the logs.

Error: Environment variable PAN_CLIENT_ID not found or empty value

Cause: defaultCredentialsProviderFactory() (and therefore autoCredentials() in memory mode) requires PAN_CLIENT_ID, PAN_CLIENT_SECRET, PAN_REFRESH_TOKEN, and PAN_ENTRYPOINT to all be present.

Solution:

1. printenv | grep PAN_ to see what your shell has.
2. If you are inside Docker, confirm the variables made it through docker run -e PAN_... or env_file:.
3. As a fallback, pass the values directly: defaultCredentialsProviderFactory({clientId, clientSecret, refreshToken, entryPoint}).

Prevention: validate all four variables at process start and exit non-zero with a clear message before constructing the SDK.

Error: not a valid JWT access token

Cause: defaultCredentialsFactory() and StaticCredentials decode the JWT to extract its exp claim. The token must have three dot-separated segments and a base64-encoded payload with a numeric exp.

Solution:

1. Verify the token has three segments: echo $TOKEN | tr '.' '\n' | wc -l should print 3.
2. Decode the middle segment: echo "<segment>" | base64 -d | jq . and confirm exp is a number.

Prevention: do not log or copy access tokens through systems that strip or wrap whitespace; even a stray newline corrupts the JWT.

Error: PanCloudError() only "tms.traps" is accepted in the EventService

Cause: EventService.filterBuilder() rejects tms.analytics, tms.config, tms.system, and tms.threat because the channel does not deliver them.

Solution: drop those tables from your filter. If you need tms.threat, query it through LoggingService instead.

Prevention: use the LogType TypeScript type for filter tables — the compiler will not stop you from passing invalid values today, but keeping the table list typed makes future SDK upgrades surface the mismatch.

Error: Property 'accTokenGuardTime' must be, at max 3300 seconds

Cause: every credentials object refuses a guard time greater than 55 minutes because Cortex access tokens are typically 1-hour bearers.

Solution: set accTokenGuardTime (or the Credentials constructor's guardTime) to a value below 3300. The default of 300 (5 minutes) is appropriate for almost all deployments.

Prevention: if you find yourself wanting longer, you are likely attempting to keep tokens alive across a long blocking call. Refactor to release the credential object during long sleeps and let the next call re-fetch.

Error: Still in RUNNING state after 20 attempts

Cause: LoggingServiceClient polls a query 20 times at 200 ms spacing. If the query is still computing, it gives up and deletes the query.

Solution:

1. For very large queries, raise retries and delay in the LoggingServiceClient constructor.
2. Switch to the auto-poll callback pattern; it has no built-in retry ceiling and runs as long as the query lives on the server.

Prevention: for queries that span months of data, narrow the startTime/endTime window or add LIMIT to the SQL.

Error: HTTP 401 mid-stream

Cause: the access token expired and the refresh attempt failed (e.g. the refresh token was revoked).

Solution:

1. Re-issue a refresh token from the Cortex hub.
2. Restart the process so autoCredentials() picks up the new value.
3. For long-running services, watch for repeated IDENTITY errors and surface them to your alerting layer; do not silently retry forever.

Prevention: rotate refresh tokens deliberately on a known cadence, not by waiting for failures. Use fsCredentialsFactory() so the new token persists across restarts without code changes.

Issue #30: Community Health Assessment

The PaloAltoNetworks/pancloud-nodejs repository tracks community-health items under issue #30. If you encounter behaviour that contradicts what this guide describes, check that issue (and the project's broader issue tracker) before filing a new ticket — the active maintainer cadence and contribution channels are documented there.

Caution

The repository's last source commit captured here is e0d9e6b from 2019-10-29. The Cortex Data Lake API surface and OAuth2 flow may have changed since then. Always cross-check endpoint behaviour against the current Cortex Data Lake reference at docs.paloaltonetworks.com before debugging unexpected responses.

7. FAQ

Q: Why is the package called pancloud in package.json but I import it as pancloud-nodejs?

The package.json name field is pancloud, version 1.0.0. The GitHub repository is pancloud-nodejs, and that is the npm install path. The TypeScript test suite imports from 'pancloud-nodejs' and that is the recommended import path for consumers.

Q: Do I need a separate @types/pancloud-nodejs package?

No. TypeScript declarations are emitted alongside the JavaScript by build:library (the tsc -p tsconfig.json step). npm install brings both.

Q: How does the SDK refresh access tokens?

Every Credentials subclass has a guardTime (default 300 seconds = 5 minutes). Before each HTTP request, the service client calls Credentials.autoRefresh(), which checks whether the cached token expires within guardTime. If yes, it calls retrieveAccessToken(), which posts the refresh token to https://api.paloaltonetworks.com/api/oauth2/RequestToken and stores the new access token. Static credentials skip this — they are no-ops.

Q: What is the difference between LoggingService and EventService?

LoggingService runs ad-hoc SQL queries against historical log indexes and returns a finite result set in batches. EventService opens a long-lived channel that emits new events as they happen, with optional filters. Use LoggingService for "show me everything between Tuesday and Thursday"; use EventService for "stream me new threat events forever".

Q: Can I run multiple service clients off the same Credentials?

Yes. The Credentials object is the auth-token cache. You can pass it into LoggingService.factory(), EventService.factory(), and DirectorySyncService.factory() simultaneously. Each client manages its own per-instance state (channel ID, query queue, stats).

Q: How do I cancel an in-flight query?

Call ls.cancelPoll(queryId) to remove it from the auto-poll queue and issue a server-side delete, or call ls.deleteQuery(queryId) directly for the same effect on the server side without touching the local queue.

Q: What does flush: true do in an Event Service filter?

When you set a filter with flush: true, the channel discards every event currently buffered server-side. Use it when you have changed the filter so significantly that old events are no longer relevant. The default is flush: false (keep buffered events).

Q: Can I use this SDK in a browser?

No. The SDK depends on Node's https, crypto, fs, and events modules. There is no browser bundle and no plan to ship one — Cortex Data Lake credentials should not live in a browser anyway.

Q: Where is the published TypeScript source?

The lib/ directory at the repository root contains the canonical TypeScript. tsc compiles it into dist/, which is what package.json points main at. There is no separate src/ versus dist/ split in the repo at commit e0d9e6b.

Q: Does the SDK validate JSON responses?

Yes — every response goes through a hand-written type guard (isJobResult, isEsFilter, isDssDomain, etc.) before it is exposed to your code. A response that does not match the expected schema raises PanCloudError(PARSER, ...).

Q: Is tms.traps the only tms.* table I can subscribe to?

For Event Service, yes. EventService.filterBuilder() validates against the invalidTables list (tms.analytics, tms.config, tms.system, tms.threat). All panw.* tables are accepted. LoggingService has no such restriction — you can query any indexed table.

Q: How do I test code that uses this SDK without a real Cortex tenant?

Use defaultCredentialsFactory(entryPoint, fakeJwt) to build a static credentials object, then mock the underlying fetch module. The SDK has a single internal fetch export (lib/fetch.ts) that wraps Node's https.request; intercepting it gives you full control over responses.

Q: How do I support multiple Cortex Data Lake tenants in one process?

Use fsCredentialsFactory() (or implement your own subclass of CortexCredentialProvider) and call registerManualDatalake() once per tenant. Each call returns its own Credentials object bound to its own datalake ID. The provider stores all of them in a single encrypted PANCLOUD_CONFIG.json and refreshes each independently.

Q: What is the L2/L3 correlation engine?

It is an optional in-memory join that pairs L3 (IP) flows with L2 (MAC) flows by sessionid. When you pass l2Corr: { timeWindow, absoluteTime, gcMultiplier } to EventService.factory(), the SDK buffers L2 and L3 events and emits a synthesized "correlated" event whenever a session appears in both — see Section 5.6 for the full pattern.