44 Commits
Author SHA1 Message Date
Quite Yellow b7f2f20f29 Merge pull request #17 from QuiteYellow/feat/dtls-clienthello-probe
Add a DTLS ClientHello probe as the liveness + diagnostic primitive
v0.1.2
2026-08-01 11:17:24 +01:00
Jack Nagy 46041bfb2c docs(readme): anti-AI writing pass
Convert em-dash prose splices to varied punctuation (periods, colons,
semicolons, commas, parens), turn **Label.**-period bullets into
**Label:** colons, drop sentence-spanning bold in "Traps to avoid", and
cut a couple of hollow intensifiers.

No content, facts, tables, code, or links changed (50/50 line diff). Left
as-is: the `## Part N —` headings (heading-anchor stability), everything
inside code/log fences, table N/A cells, and numbered-list
`**Bold** — desc` carve-outs.
2026-08-01 11:10:58 +01:00
Jack Nagy 1a35cd59a1 feat(protocol): add DTLS ClientHello liveness probe + wire it into the bridge
A stateless-by-default DTLS ClientHello probe that classifies a host:port
as DEAD/LIVE/COMPLETED/REJECTED in ~1 RTT off the server's first flight,
sitting in front of the full handshake.

Probe (smartthings_local/protocol/dtls_probe.py):
- Stateless liveness mode (default): stops at HelloVerifyRequest and never
  sends the cookie'd second ClientHello, so by RFC 6347 §4.2.1 it leaves
  no association on the device — safe to run before a real connect.
- Diagnostic mode (stateless=False): drives the handshake further to
  capture cipher/cert-chain/CertificateRequest or a fatal Alert, for
  OCF-PKI-wall characterization (#16). Kept out of hot reconnect paths.
- Retransmit + retries: services OpenSSL's DTLS retransmit timer so a
  single dropped ClientHello no longer reads as a false DEAD.

MQTT bridge (mqtt_demo):
- Stateless pre-flight gate in session_once() rejects a silent/rebooting
  device or wrong port in ~3s (retries=1) instead of eating the 12s
  HANDSHAKE_TIMEOUT_S per reconnect.
- OCF-band port autodiscovery when OCF_PORT is unset: races the band in
  parallel and returns on the first port to answer LIVE (~1 RTT, abandoning
  the dead-port probes), cached across reconnects; the stateless gate
  leaves no orphan, preserving the fixed-source-port §4.2.8 invariant.

Validated on real hardware (dryer 49155 / oven 49154): parallel discovery
resolves both ports in <1s, connect with no orphan cooldown, and a wrong
pinned port rejected in ~3s.

Tests: probe behaviour (retransmit recovery, stateless single-flight
guard, silent-port flight budget, diagnostic continuation) and bridge
port-resolution (pinned gate, parallel discovery early-exit, cache).
2026-08-01 10:57:54 +01:00
Jack Nagyandvmvarga 8c2108a510 feat(protocol): fixed DTLS source port so reconnects evict orphaned sessions
Root-cause fix for the stale-session stall on always-on appliances (#14).
When the client dies without close_notify (crash, SIGKILL), the device
keeps an orphaned DTLS association keyed to the old 5-tuple; a reconnect
from a fresh ephemeral port presents as a brand-new peer, so the orphan
lingers until the device's own timer reaps it (observed 5-15 min).

RFC 6347 §4.2.8 covers exactly this: a ClientHello arriving on an existing
association's 5-tuple means the peer rebooted, and the server must complete
the new handshake and discard the old association. Add an optional
local_port to DtlsCoapSession that binds the UDP source port, and have the
bridge bind base+appliance-index, so every reconnect re-handshakes over the
same 5-tuple and the orphan is evicted instead of waited out.

Bench-verified on live hardware (2026-07-26): RT-OCF accepts the
same-5-tuple rehandshake (oven, dryer: handshake completes over a
crash-orphaned association, reads work immediately). The oven does not
reproduce the fridge stall even with 11 crash-orphaned OBSERVE
registrations, so fridge-side confirmation of the eviction is still needed.

Co-authored-by: vmvarga <garrysuchiy@gmail.com>
v0.1.1
2026-07-26 20:53:56 +01:00
Jack Nagy 6653de3b2c docs(readme): refine tested-combos after PR #13
- credit @indykoning + note localthings test path for the washer row
- soften DV90T mnid grouping (mnid=0AJT confirmed on DV5000T only)
- de-speculate the same-family note now that a washer is confirmed
2026-07-26 18:44:42 +01:00
Quite Yellow a2760eaa11 Merge pull request #13 from indykoning/patch-1
Added tested machines
2026-07-26 18:43:28 +01:00
indykoning 231d2b5f1a Added tested machines 2026-07-23 13:19:05 +02:00
Jack Nagy 072af1bfa1 docs(readme): reframe around the smartthings-local library
- Lead with the pip-installable library; frame the MQTT bridge as a
  reference demo. Add a library quick-start (install, DtlsCoapSession
  example, in-memory cert_pem/key_pem variant).
- Fix stale protocol/ + ocf/ references to smartthings_local/*; update
  the repo-layout tree (nested package, ocf_root_ca.pem, pyproject.toml,
  tests/, publish.yml); drop the non-existent auth.py.
- Correct the write-surface trap: reconciliation is a deferred poll, not
  a post-write fetch-back (which itself triggered Samsung's revert).
  Distinguish hardware-gated parity (power/child-lock/RC-enable) from
  the open oven remote-start problem.
- Note the few write surfaces the cloud HA integration doesn't expose
  (dryer course, oven setpoint). Drop the achieved collaborators-wanted
  callout.
2026-07-07 19:41:06 +01:00
Quite Yellow c46dda9766 Merge pull request #12 from mbillow/cert-pem-support
Support in-memory PEM cert/key alongside file paths in DtlsCoapSession
v0.1.0
2026-07-07 17:33:15 +01:00
Marc Billow a2dc524c0b feat: support in-memory PEM cert/key alongside file paths in DtlsCoapSession
localthings mints its client cert at runtime through the HA config flow
and never writes it to disk. DtlsCoapSession only accepted cert_path/
key_path (file-based), which would have forced localthings to write its
in-memory cert/key to disk on every connect just to migrate off its
vendored copy of this transport layer.

Adds an alternate cert_pem/key_pem constructor path (ported from
localthings' own _load_pem_chain), validated so exactly one cert source
(file pair or PEM pair) is required. Existing file-path callers
(mqtt_demo, setup_cert.py) are unaffected — verified against both real
appliances with each constructor path.
2026-07-06 15:04:20 -05:00
Jack Nagy 3fdc735141 feat(packaging): nest protocol/ + ocf/ under smartthings_local, add PyPI packaging
Nest the two library packages under a single import namespace so they
can ship as one distribution:

  protocol/ -> smartthings_local/protocol/
  ocf/      -> smartthings_local/ocf/   (git mv, history preserved)

- Rewrite all imports protocol.* -> smartthings_local.protocol.*,
  ocf.* -> smartthings_local.ocf.* across the ocf modules, mqtt_demo/
  (bridge, descriptor, samples), and tests.
- Add pyproject.toml: dist name `smartthings-local`, hatch-vcs versioning
  from v* tags, wheel ships only smartthings_local/.
- Add .github/workflows/publish.yml: build + PyPI Trusted Publishing on
  v* tags (OIDC, no stored token).
- Force-include protocol/ocf_root_ca.pem via [tool.hatch.build] artifacts:
  it is tracked but matches .gitignore's *.pem, so hatchling's VCS file
  selection would drop it — and dtls_session.py loads it at runtime.
- Update mqtt_demo Dockerfile COPY and deploy.sh tar allowlist to the
  single smartthings_local/ package.
- .gitignore: build artifacts (_version.py, dist/, *.egg-info/).

Validated: pytest tests/ (11 passed), python -m build produces sdist +
wheel with the pem bundled, fresh pip install resolves all nested imports
with the pem readable from site-packages.
2026-07-06 20:29:19 +01:00
Quite Yellow 9004bac729 Merge pull request #8 from mbillow/protocol-library-reorg
Reorg into protocol/ + ocf/ + mqtt_demo/, port 3 DTLS reliability fixes
2026-07-06 17:33:00 +01:00
Marc Billow ce6985ca3f fix: restore Block2 retry debug logging dropped during port
Task 8's port of retry/retransmit from localthings dropped the per-attempt
timeout/retry log lines (present in coap_dtls.py) — found while gathering
real-device pacing evidence, where the missing logs made retry frequency
impossible to see. Restores both the retry and final-timeout log lines.
2026-07-05 16:51:44 -05:00
Marc Billow ff688f5d5d fix: residual inter-request pacing instead of blind full-interval sleep
pace() slept the entire rate-limit interval every call regardless of how
much of it had already elapsed since the last real send (e.g. spent
processing the previous block's response). Track the last send timestamp
and only sleep the remainder, recovering time already spent between
requests without changing the enforced minimum interval.

Verified against 10.0.0.129/10.0.0.254: multi-block /device/0 fetches
still complete cleanly, faster per sweep, with the same request spacing
guarantee (patch and measurement from QuiteYellow, PR #8 review).
2026-07-05 16:51:17 -05:00
Marc Billow 27480dcb1d fix: pace inter-tier polling; fix stale package-rename references in docs 2026-07-04 17:29:22 -05:00
Marc Billow 3a2d3da3b2 docs: point HA users at localthings, freeze mqtt_demo/samples as reference-only 2026-07-04 17:29:22 -05:00
Marc Billow 70baa3baf0 fix: rate-limit inter-request pacing to avoid RT-OCF request drops 2026-07-04 17:29:22 -05:00
Marc Billow 58df1b242f fix: retry Block2 GETs on timeout, track server-negotiated block size
Ported from localthings fork: bounded per-block retransmission
(_BLOCK_MAX_ATTEMPTS/_BLOCK_ACK_TIMEOUT) instead of a single attempt
per block, SZX renegotiation tracking, and a bugfix moving
conn.send(datagram) inside the try that catches its own SSL errors.
2026-07-04 17:29:22 -05:00
Marc Billow 0fc65339b5 fix: validate device cert chain against OCF root CA instead of VERIFY_NONE 2026-07-04 17:29:22 -05:00
Marc Billow c7232b3df6 fix: isolate subprocess PYTHONPATH in import-isolation test to prevent false pass 2026-07-04 17:29:22 -05:00
Marc Billow 8a6bc8d594 test: verify protocol/ + ocf/ import in isolation from mqtt_demo/
- Add test_import_isolation.py: isolation test that copies protocol/ and ocf/
  to a temp directory without mqtt_demo/ and verifies all modules import
- Add conftest.py: pytest configuration to add repo root to sys.path,
  enabling tests to import protocol/ and ocf/ packages
2026-07-04 17:29:22 -05:00
Marc Billow 4789ac8f55 fix: include .dockerignore in deploy.sh's remote tar package to prevent .env leaking into the image 2026-07-04 16:22:31 -05:00
Marc Billow 5546454097 fix: remove ocf/state_cache.py's stale mqtt_demo-descriptor type dependency 2026-07-04 16:17:44 -05:00
Marc Billow f573dddb14 fix: prevent .env from being baked into mqtt_demo image; fix dangling requirements-bootstrap.txt path 2026-07-04 16:15:44 -05:00
Marc Billow aa9ec0f0d9 refactor: extract mqtt_demo/ — move bridge, config, logger, appliance descriptors 2026-07-04 16:11:10 -05:00
Marc Billow 24fc197a9e refactor: extract ocf/ — fold sensors.index_links into StateCache.index_device_tree 2026-07-04 16:02:08 -05:00
Marc Billow 10773bb8c8 refactor: extract protocol/ — split CoAP wire helpers from DtlsCoapSession 2026-07-04 15:55:12 -05:00
Marc Billow d33b46171a test: characterize CoAP wire encode/decode before extracting protocol/ 2026-07-04 15:50:38 -05:00
Marc Billow cae0592324 test: add pytest as a dev dependency 2026-07-04 15:49:39 -05:00
Quite Yellow 3cdfc6a3e6 Merge pull request #7 from QuiteYellow/artik-fridge
Merge PR #1: ARTIK051 fridge + session-recovery robustness
2026-07-02 21:24:28 +01:00
Jack Nagy b34f0e8248 Docs: fridge notes + firmware-family limitation callout
- Add ARTIK051_REF_17K row to the tested-combinations table with a
  link to aminorjourney's PR.
- New 'Firmware families — a limitation' section under Part 1
  explaining that descriptors are firmware-family-specific with no
  runtime feature detection, so the wrong descriptor produces
  half-broken sensors rather than a clean error.
- New 'Fridge (ARTIK051)' section under Per-appliance notes with the
  capability table + firmware-specific observations (port 49155,
  minimal /oic/res, vestigial /hass paths, collection-resource door
  model vs newer per-instance-resource fridges).
- Update config-keys reference: CLASS list gains 'fridge',
  OCF_PORT defaults list gains fridge=49155.
2026-07-02 21:22:00 +01:00
Aminorjourney 015e4a4e9e Session-recovery robustness (from PR #1)
- Suppress ConnectionError log noise from in-flight requests draining
  after a session close (poll_scheduler + keepalive log at DEBUG).
- Null KeepaliveTask.on_unreachable during a forced reconnect so the
  dying session's keepalive can't flip HA availability offline after
  the new session is already healthy. Ported into PR #5's
  _maybe_force_reconnect flow.

Drops the _session_stop / publish-health force-close mechanism from
the original PR — PR #5's last_success_ts + _maybe_force_reconnect
already covers the 'session dead, restart it' goal via a different
path, and running both means two paths force-closing the same session
on the same failure.
2026-07-02 21:15:00 +01:00
Aminorjourney 93f78f99a2 feat: add Samsung ARTIK051_REF_17K fridge-freezer descriptor
- Full resource map discovered via live CoAP-DTLS session against
  ARTIK051_REF_17K fridge-freezer (firmware DA-REF-ART-COMMON-1_20201124)
- seed_path: /device/0 (returns 32 links including hidden appliance resources)
- Entities: fridge/freezer temp + setpoints, 3x door sensors (fridge,
  freezer, convertible zone), power (W), energy (kWh, total_increasing),
  water filter usage/status, ice maker, rapid fridge/freeze, sabbath mode
- Port: 49155 (not 49154)

First known public documentation of this firmware's local CoAP-DTLS
resource layout.
2026-07-02 21:10:00 +01:00
Jack Nagy e4c6702c48 Add MIT license 2026-07-01 20:30:48 +01:00
Jack Nagy e9a30d96f9 Drop dangling local-tools/ references from README 2026-06-30 19:51:36 +01:00
Quite Yellow 788408da9a Update README.md 2026-06-30 19:49:02 +01:00
Quite Yellow 27a614fcb8 Merge pull request #5 from QuiteYellow/production-hardening
Production-hardening pass: half-open detection, cascade throttle, OBSERVE refresh, lamp/door coupling
2026-06-30 19:36:09 +01:00
Quite Yellow 19946e21d7 Merge pull request #3 from QuiteYellow/cert-refactor
Ship setup_cert.py to repo root, auto-fetch all CA materials
2026-06-30 19:33:41 +01:00
Jack Nagy 36acf3a209 Production-hardening pass: half-open detection, cascade throttle, OBSERVE refresh, lamp/door coupling
Four failure modes observed in-house since the polling-first refactor
(709fdf4):

1. Half-open DTLS sessions where the socket stays writable but the peer
   has gone silent. Ping sends succeed against a wedged peer because
   RT-OCF doesn't reliably emit a RST; only successful polls prove the
   session is live.

   - PollScheduler exposes last_success_ts (bumped on every 2.05).
   - KeepaliveTask takes liveness_fn(); ticks fail if no 2.05 in the
     last 60s, even when the ping send succeeded.
   - Bridge force-closes the session after 120s unreachable so
     run_forever() breaks out of sess.join() and reconnects.

2. RT-OCF cascade under load. One wedged path can eat 8s of timeout,
   the next tier tick fires immediately and stacks another attempt,
   and the device wedges harder.

   - PollTier.timeout_s per-tier override (hot=2s, warm=4s, sweep=15s).
   - On TimeoutError, the href goes into a 5-60s cooldown via the
     existing _defer_until mechanism.
   - take_window_stats() now reports successful-poll RTT separately
     from a timeout count, exposed as the "Poll Timeouts (window)"
     diagnostic entity in HA.
   - Active-window throttle: if the previous health window saw >=3
     timeouts and is_active=True, drop back to idle cadence -- stops
     stacking polls on a stalled responder.

3. OBSERVE table aging across cloud-auth blips. The device stays
   DTLS-reachable but the on-device stack clears its observer table
   during the blip, so push delivery stays dead even after upstream
   recovers.

   - New ObserveRefreshTask per bridge; every 6h derregs all current
     observer tokens and re-subscribes on the existing session.

4. Oven lamp/door coupling. Oven hardware auto-drives the lamp from
   door state but /mode/vs/0 is warm-tier (30s) so HA showed stale lamp
   during a cook.

   - Track door + lamp value-change timestamps in descriptor_state.
     When the door transition is newer, derive lamp from door_open.
     When an HA optimistic write is newer, the cache value wins.

Also: ANSI-coloured WARNING/ERROR lines (NO_COLOR=1 opt-out), jittered
reconnect backoff so dryer + oven don't sync up after a router blip.

In-house verification: running on dryer + oven since 2026-06-03.
2026-06-30 19:27:59 +01:00
Jack Nagy 0374f8cf68 Ship setup_cert.py to repo root, auto-fetch all CA materials
Closes #2.

Previously the cert minting script lived in local-tools/ (gitignored)
and the README pointed at a cert-only source that didn't include the
private key or upstream chain.

setup_cert.py now lives at the repo root and live-fetches both the
peer UUID (from the relevant TLS server cert subject DN) and the
full AC14K_M + upstream chain bundle (RemoteAccessCA + CECA + ROOTCA)
from a public mirror. Each fetch has an inline workaround if the
network is restricted (UUID=..., AC14K_M_CERT_BUNDLE=...,
BRAYSTORM_URL=...). Modulus-pair check catches a wrong-key mistake
before signing. bootstrap.py removed -- imported a package that was
renamed in commit 709fdf4.

Output files use neutral client.* names. README, .env.example,
docker-compose.yml, deploy.sh, and config.py updated to match.

Provenance receipts in local-tools/cert_provenance.md.
2026-06-30 19:27:24 +01:00
Jack Nagy 709fdf444d Refactor to polling-first architecture with OBSERVE as accelerator
State freshness now comes from a tiered PollScheduler over the persistent
DTLS session; OBSERVE registrations are kept as an opportunistic
acceleration layer. Behaviour is identical online vs air-gapped except
for worst-case freshness latency.

Adds three modules:
- StateCache: single source of truth, source-tagged change events
- PollScheduler: hot/warm/cold + sweep tiers, write-defer past the
  fetchback-revert window, per-window RTT/slow-poll tracking
- KeepaliveTask: CoAP empty-CON ping with consecutive-fail detection
  driving MQTT availability

Bridge publishes per-appliance diagnostic entities (Push Active, Last
Update Source, Poll Max RTT, Slow Polls, Poll Errors, Stalest Resource
Age, Last OBSERVE Age) under HA's Diagnostic section. Tier cadences
are descriptor-declared, calibrated against measured per-firmware
ceilings (dryer ~14 req/s, oven ~8 req/s via probe_poll_rate_combined.py).

Drops HEARTBEAT_INTERVAL_S in favour of the descriptor-declared sweep
tier; PING_INTERVAL_S now consumed by KeepaliveTask inside the bridge
rather than driven from main.py.

README explains the push/poll split and what happens when the appliance
is blocked from internet.
2026-06-03 18:38:04 +01:00
Jack Nagy 8775f7a930 Add oven controls and fix Samsung-OCF write semantics
Major session of local-OCF reverse engineering against the NV7000BS
oven and DV5000T dryer. Surfaces a working set of HA entities for the
oven and resolves several Samsung-quirk regressions in the bridge's
write path.

Key behavioural fixes:
- OBSERVE registrations now use single-byte tokens. Samsung RT-OCF
  silently drops registrations with TKL>1; same 4-byte tokens work
  fine for GET/POST. Symptom was that writes returned 2.04 but the
  appliance never pushed state changes.
- Per-session random starting tokens + MID. Samsung retains observer
  state across DTLS reconnects from the same cert; reusing tokens on
  reconnect silently no-ops.
- OBSERVE deregister sent on DtlsCoapSession.close(), with a stop-
  watcher thread in PushBridge.session_once() so SIGINT/SIGTERM
  actually reaches close() instead of hanging in sess.join().
- pyOpenSSL is not thread-safe — reader-loop conn.* calls now hold
  the same _send_lock the sender uses, dispatching decrypted packets
  outside the lock so the auto-ACK send doesn't deadlock.
- Periodic CoAP Ping (RFC 7252 §4.4) keepalive to keep DTLS warm.
- Post-write Block2 fetchback REMOVED. It was the root cause of
  every "setpoint/operationTime/modes revert ~3s after write"
  symptom — Samsung's stack treats a read on a freshly-written
  resource as a signal to invalidate that write. OBSERVE pushes
  keep HA in sync without the verification GET.

HA-facing changes (oven):
- New entities: Lamp (light), Sound (switch), Fast preheat, Natural
  steam, Setpoint (number), Cook time (number), Stop cycle (button).
- Cooking mode surfaced as a read-only sensor — the oven owns the
  modes field once a cycle is active and rolls local writes back.
- Cook time writes operationTime + remainingTime on
  /operational/state/vs/0 (discovered via OBSERVE capture of
  SmartThings mid-cycle changes — UpperTimerSet on /mode/vs/0
  options is vestigial and doesn't drive the running cycle).
- New cycle_active MQTT availability topic. Writes the oven only
  honours mid-cycle (setpoint, cook time, fast preheat, natural
  steam, stop) gate on it via avail_with_cycle / avail_with_remote_
  and_cycle. Sound + Lamp remain always-available.
- Cycle Start deliberately NOT exposed. Every byte-level approxi-
  mation of SmartThings's working start sequence is rejected at
  the firmware level. Empty discovery payloads remove the previous
  Start button and Cooking-mode select cleanly from HA.

Diagnostics:
- DEBUG_BRIDGE=1 env var enables verbose tracing (rx CON/NON/ACK/
  RST per frame, full link-tree dump at seed, /oic/res directory,
  REP changes on /operational/state, /oven, /power, mode options).
  Quiet in production.
2026-05-31 20:43:47 +01:00
Quite Yellow 85580d81b7 Update README.md 2026-05-31 15:51:43 +01:00
Jack Nagy c99ef324fc Initial commit 2026-05-31 15:50:15 +01:00