From 05445ef83138331bcc5c303cb1837c12033f52b7 Mon Sep 17 00:00:00 2001 From: Jake Jarvis Date: Wed, 24 Sep 2025 19:19:41 -0400 Subject: [PATCH] Update repository guidelines and README: enhance project structure documentation, clarify build and test commands, and improve API usage examples for better user understanding. --- AGENTS.md | 52 +++++++++--------- README.md | 148 +++++++++++++++++++++++++++++++++++++++++++++++---- src/types.ts | 1 - 3 files changed, 165 insertions(+), 36 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0c0fe2f..3336928 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,37 +1,39 @@ # Repository Guidelines ## Project Structure & Module Organization -- `src/api/`: Public lookup orchestration (`lookup.ts`). -- `src/rdap/`: RDAP bootstrap, client, and normalization (`bootstrap.ts`, `client.ts`, `normalize.ts`). -- `src/whois/`: WHOIS TCP client, discovery, referral, normalization, catalog. -- `src/lib/`: Shared utilities (`dates.ts`, `async.ts`, `domain.ts`, `text.ts`). -- `src/types.ts`: Public types. `src/index.ts` re-exports API and types. -- Tests: per-module `__tests__/` folders with `*.test.ts` (e.g., `src/lib/__tests__/dates.test.ts`). -- `dist/`: Build output (generated). Do not edit. -- `cli.mjs`: Local CLI for manual checks. +- Source: `src/` (entry: `src/index.ts`). +- Protocol modules: `src/rdap/` and `src/whois/` (clients, normalize, helpers). +- Utilities: `src/lib/`, shared types: `src/types.ts`. +- Tests live beside code in `__tests__/` and use `*.test.ts`. +- Built output: `dist/` (ESM + types). Quick CLI: `cli.mjs` for manual checks. ## Build, Test, and Development Commands -- `npm run build`: Clean and compile with `tsc -p tsconfig.build.json` (excludes tests); outputs to `dist/`. -- `npm test`: Compile tests, then run Node’s test runner on `dist/**/*.test.js`. -- `npm run lint`: Biome format+lint with autofix per `biome.json`. -- Example CLI: `npm run build && node cli.mjs example.com`. +- `npm run build` — clean and compile TypeScript to `dist/`. +- `npm test` — type-check, build, and run Node’s test runner on `dist/**/*.test.js`. +- `npm run lint` — format and lint with Biome (auto-fixes). +- Manual smoke: `node cli.mjs example.com` (after `npm run build`). +- Network smoke tests (opt‑in): `SMOKE=1 npm test`. ## Coding Style & Naming Conventions -- TypeScript strict; ES2022 ESM (`tsconfig.json`). -- Biome-enforced: spaces indentation; double quotes; organized imports. -- Filenames: kebab-case for modules (e.g., `normalize-rdap.ts`). -- Identifiers: camelCase; avoid abbreviations; explicit return types for exported functions. +- TypeScript strict, ESM (`module`/`moduleResolution: NodeNext`). +- Formatting via Biome: spaces, 2‑space indent, double quotes. +- Naming: functions/vars `camelCase`, types/interfaces `PascalCase`, constants `UPPER_SNAKE_CASE`. +- Files: lowercase; tests in `__tests__/` with `*.test.ts`. +- Prefer named exports; avoid default exports unless ergonomic. ## Testing Guidelines -- Framework: Node `node:test`. -- Tests live under `src/**/__tests__` and are deterministic/offline by default. -- Smoke tests gated by `SMOKE=1` (e.g., `SMOKE=1 npm test`). -- Run all tests: `npm test`. +- Framework: Node `node:test` + `assert/strict`. +- Unit tests must be deterministic and offline. Gate network tests behind `SMOKE=1`. +- Place tests near the code (e.g., `src/whois/__tests__/normalize.test.ts`). +- Run locally: `npm test`; for smoke: `SMOKE=1 npm test`. ## Commit & Pull Request Guidelines -- Commits: imperative, concise summaries (e.g., “Refactor lookup: tighten error handling”). -- PRs: include what/why, linked issues, and test notes; ensure `npm run lint && npm test` pass. +- Commits: concise imperative subject (e.g., “Add WHOIS referral fallback”), with a brief “what/why” body. +- Scope changes narrowly; keep diffs readable. +- PRs: clear description, linked issues, test plan (commands + expected output), and any CLI screenshots/logs when relevant. +- Required checks before PR: `npm run lint && npm test`. -## Release & Security Notes -- Publish only `dist/`; `prepublishOnly` runs the build. Tests are excluded via `tsconfig.build.json` and `files` in `package.json`. -- Node >= 18.17 with global `fetch`. WHOIS uses TCP 43; be mindful of registry rate limits. +## Security & Configuration Tips +- Node `>= 18.17`. No external HTTP client; uses global `fetch` and TCP 43 for WHOIS. +- Avoid hardcoding endpoints. Respect options: `timeoutMs`, `followWhoisReferral`, `rdapOnly`, `whoisOnly`. +- Do not run network tests in CI by default; use `SMOKE=1` only when appropriate. diff --git a/README.md b/README.md index 7c13187..4d2c176 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,13 @@ -# rdapper +# 🎩 rdapper -🤵 Fetch and parse domain registration data using RDAP and falling back to WHOIS. +RDAP‑first domain registration lookups with WHOIS fallback. Produces a single, normalized record shape regardless of source. + +- RDAP discovery via IANA bootstrap (`https://data.iana.org/rdap/dns.json`) +- WHOIS TCP 43 client with TLD discovery, registrar referral follow, and curated exceptions +- Normalized output: registrar, contacts, nameservers, statuses, dates, DNSSEC, source metadata +- TypeScript types included; ESM‑only; no external HTTP client (uses global `fetch`) + +Requirements: Node >= 18.17 (global `fetch`). WHOIS uses TCP port 43. ## Install @@ -8,25 +15,146 @@ npm install rdapper ``` -## Usage +## Quick Start ```ts import { lookupDomain } from "rdapper"; const { ok, record, error } = await lookupDomain("example.com", { - timeoutMs: 15000, - followWhoisReferral: true, + timeoutMs: 15000, // default 15000 + followWhoisReferral: true, // default true }); if (!ok) throw new Error(error); -console.log(record); +console.log(record); // normalized DomainRecord ``` -## Notes +Also available: -- Uses IANA RDAP bootstrap and RDAP JSON when available; falls back to WHOIS. -- Standardized output regardless of source. -- No external HTTP client deps; relies on global fetch. WHOIS uses TCP 43. +```ts +import { isRegistered, isAvailable } from "rdapper"; + +await isRegistered("example.com"); // => true +await isAvailable("likely-unregistered-thing-320485230458.com"); // => false +``` + +## API + +- `lookupDomain(domain, options?) => Promise` + - Tries RDAP first for the domain’s TLD; if unavailable or fails, falls back to WHOIS unless toggled off. + - Result is `{ ok: boolean, record?: DomainRecord, error?: string }`. +- `isRegistered(domain, options?) => Promise` +- `isAvailable(domain, options?) => Promise` + +### Options + +- `timeoutMs?: number` – Total timeout budget per network operation (default `15000`). +- `rdapOnly?: boolean` – Only attempt RDAP; do not fall back to WHOIS. +- `whoisOnly?: boolean` – Skip RDAP and query WHOIS directly. +- `followWhoisReferral?: boolean` – Follow registrar referral from the TLD WHOIS (default `true`). +- `customBootstrapUrl?: string` – Override RDAP bootstrap URL. +- `whoisHints?: Record` – Override/add authoritative WHOIS per TLD (keys are lowercase TLDs, values may include or omit `whois://`). +- `signal?: AbortSignal` – Optional cancellation signal. + +### `DomainRecord` schema + +The exact presence of fields depends on registry/registrar data and whether RDAP or WHOIS was used. + +```ts +interface DomainRecord { + domain: string; // normalized name (unicode when available) + tld: string; // terminal TLD label (e.g., "com") + isRegistered: boolean; // availability heuristic (WHOIS) or true (RDAP) + isIDN?: boolean; // uses punycode labels (xn--) + unicodeName?: string; // RDAP unicodeName when provided + punycodeName?: string; // RDAP ldhName when provided + registry?: string; // registry operator (rarely available) + registrar?: { + name?: string; + ianaId?: string; + url?: string; + email?: string; + phone?: string; + }; + reseller?: string; + statuses?: Array<{ status: string; description?: string; raw?: string }>; + creationDate?: string; // ISO 8601 (UTC) + updatedDate?: string; // ISO 8601 (UTC) + expirationDate?: string; // ISO 8601 (UTC) + deletionDate?: string; // ISO 8601 (UTC) + transferLock?: boolean; // derived from EPP statuses + dnssec?: { + enabled: boolean; + dsRecords?: Array<{ keyTag?: number; algorithm?: number; digestType?: number; digest?: string }>; + }; + nameservers?: Array<{ host: string; ipv4?: string[]; ipv6?: string[] }>; + contacts?: Array<{ + type: "registrant" | "admin" | "tech" | "billing" | "abuse" | "registrar" | "reseller" | "unknown"; + name?: string; organization?: string; email?: string | string[]; phone?: string | string[]; fax?: string | string[]; + street?: string[]; city?: string; state?: string; postalCode?: string; country?: string; countryCode?: string; + }>; + whoisServer?: string; // authoritative WHOIS queried (if any) + rdapServers?: string[]; // RDAP base URLs tried + rawRdap?: unknown; // raw RDAP JSON (RDAP source) + rawWhois?: string; // raw WHOIS text (WHOIS source) + source: "rdap" | "whois"; // which path produced data + fetchedAt: string; // ISO 8601 timestamp + warnings?: string[]; +} +``` + +### Example output + +```json +{ + "domain": "example.com", + "tld": "com", + "isRegistered": true, + "registrar": { "name": "Internet Assigned Numbers Authority", "ianaId": "376" }, + "statuses": [{ "status": "clientTransferProhibited" }], + "nameservers": [{ "host": "a.iana-servers.net" }, { "host": "b.iana-servers.net" }], + "dnssec": { "enabled": true }, + "source": "rdap", + "fetchedAt": "2025-01-01T00:00:00Z" +} +``` + +## How it works + +- RDAP + - Discovers base URLs for the TLD via IANA’s RDAP bootstrap JSON. + - Tries each base until one responds successfully; parses standard RDAP domain JSON. + - Normalizes registrar (from `entities`), contacts (vCard), nameservers (`ipAddresses`), events (created/changed/expiration), statuses, and DNSSEC (`secureDNS`). +- WHOIS + - Discovers the authoritative TLD WHOIS via `whois.iana.org` (TCP 43), with curated exceptions for tricky zones and public SLDs. + - Queries the TLD WHOIS; if a registrar referral is present and `followWhoisReferral !== false`, follows one hop to the registrar WHOIS. + - Normalizes common key/value variants across gTLD/ccTLD formats (dates, statuses, nameservers, contacts). Availability is inferred from common phrases (best‑effort heuristic). + +Timeouts are enforced per request using a simple race against `timeoutMs` (default 15s). All network I/O is performed with global `fetch` (RDAP) and a raw TCP socket (WHOIS). + +## Development + +- Build: `npm run build` +- Test: `npm test` + - By default, tests are offline/deterministic. + - Smoke tests that hit the network are gated by `SMOKE=1`, e.g. `SMOKE=1 npm test`. +- Lint/format: `npm run lint` (Biome) + +Project layout: + +- `src/rdap/` – RDAP bootstrap, client, and normalization +- `src/whois/` – WHOIS TCP client, discovery/referral, normalization, exceptions +- `src/lib/` – utilities for dates, text parsing, domain processing, async +- `src/types.ts` – public types; `src/index.ts` re‑exports API and types +- `cli.mjs` – local CLI helper for quick testing + +## Caveats + +- WHOIS text formats vary significantly across registries/registrars; normalization is best‑effort. +- Availability detection relies on common WHOIS phrases and is not authoritative. +- Some TLDs provide no RDAP service; `rdapOnly: true` will fail for them. +- Registries may throttle or block WHOIS; respect rate limits and usage policies. +- Field presence depends on source and privacy policies (e.g., redaction/withholding). ## License diff --git a/src/types.ts b/src/types.ts index 3d13ed7..50d4006 100644 --- a/src/types.ts +++ b/src/types.ts @@ -87,7 +87,6 @@ export interface LookupOptions { customBootstrapUrl?: string; // override IANA bootstrap // WHOIS discovery and query tuning whoisHints?: Record; // override/add authoritative WHOIS per TLD - maxWhoisHops?: number; // max referral hops to follow (default 2) signal?: AbortSignal; }