Skip to content

Strengthen Java-Thread-Affinity native integration, documentation, and quality gates#192

Open
peter-lawrey wants to merge 33 commits into
developfrom
adv/develop
Open

Strengthen Java-Thread-Affinity native integration, documentation, and quality gates#192
peter-lawrey wants to merge 33 commits into
developfrom
adv/develop

Conversation

@peter-lawrey

@peter-lawrey peter-lawrey commented Nov 21, 2025

Copy link
Copy Markdown
Member

This PR hardens the Java-Thread-Affinity module across native code, affinity selection, and timing paths, while adding explicit architecture docs and quality profiles. It also introduces shared AGENTS.md guidance for AI and human contributors, and a Claude-specific helper file.


Functional changes

  • Affinity selection and JNA fallback

    • Refactored Affinity initialiser to:

      • Guard all native initialisation with a try/catch and fall back cleanly to NullAffinity on failure.
      • Use a thread-safe, lazy isJNAAvailable() check that reflects the actual JNA presence and version.
      • Ensure getAffinity() always returns a valid implementation (falls back to NullAffinity if needed).
  • Native JNI affinity and timer behaviour

    • Hardened NativeAffinity JNI layer:

      • Added getVersion0() and surfaced it as NativeAffinity.VERSION, initialised at class-load time and logged when the native library is present.
      • Replaced C++ exceptions with Java UnsupportedOperationException / RuntimeException via JNI, including error handling for sched_setaffinity failures and unsupported platforms.
      • Added JNI_OnLoad returning JNI_VERSION_1_8 to make version negotiation explicit.
    • Strengthened JNIClock:

      • Adjusted estimateFrequency loop to always invoke rdtsc0() during spin, avoiding the JIT optimising away calls.
      • Clarified rdtsc0 as a static native method.
  • Native build pipeline

    • Reworked affinity/src/main/c/Makefile:

      • Added VERSION and JAVA_HOME parameters (wired from Maven exec-maven-plugin) and propagate them via -DPROJECT_VERSION for getVersion0().
      • Introduced hardened CFLAGS/CXXFLAGS (optimisation, warnings, stack protector, FORTIFY, PIE).
      • Split compilation into object files for each native source and linked them into libCEInternals.so.
      • Ensured macOS-specific source is included only on Darwin and switched sprintf to snprintf.
      • Added clean target that removes both the library and object files.
  • Locking and inter-process coordination

    • Updated FileLockBasedLockChecker:

      • Reworked getInstance() to return a fresh instance, avoiding long-lived shared state in tests.
      • Switched all file IO to UTF-8, with better error messages when temp directories cannot be created.
      • Added documentation explaining lock-file semantics and retry behaviour.
    • Tightened LockCheck helper methods (storePid, isLockFree) by making the synchronized flag placement conventional and explicit.

  • JNA-based affinity helpers

    • Enriched LinuxHelper, LinuxJNAAffinity, PosixJNAAffinity, WindowsJNAAffinity, OSXJNAAffinity, SolarisJNAAffinity, Utilities, and VersionHelper with class-level Javadoc capturing:

      • Behavioural role (CPU mask handling, pid/tid resolution, libc calls).
      • Thread-local caching semantics for thread IDs.
      • Version parsing and comparison rules.
    • Normalised ThreadLocal naming (THREAD_IDthreadId) to better reflect intent and avoid Checkstyle noise.

    • Ensured LinuxJNAAffinity and PosixJNAAffinity handle missing native helpers gracefully (do not throw from static initialisers, rely on LOADED flags).

  • Ticker and affinity APIs

    • Documented BootClassPath, LockInventory, ticker packages, and affinity packages with package-info.java:

      • Public APIs in net.openhft.affinity and net.openhft.ticker now describe purpose, scope, and usage patterns (thread pinning, low-jitter time sources).
      • Internal packages (impl, lockchecker, main, ticker.impl) are explicitly marked as implementation/support code.
    • Adjusted MicroJitterSampler busy-wait loop to use an empty while body without a trailing semicolon in the same line, clarifying intent.

  • Tests and safety nets

    • Added/updated tests to cover new behaviour:

      • AffinitySelectionAndFallbackTest and AffinityJnaUnavailableSimulationTest verify default selection on Linux and fallback to NullAffinity when JNA is genuinely absent.

      • LinuxAffinityParityTest exercises JNI vs JNA affinity parity on Linux, validating that masks intersect correctly when set via either path.

      • JNIClockBasicBehaviourTest and updated JNIClockTest validate tick changes, monotonic nanoTime behaviour, concurrent access, and tolerance bounds compared to System.nanoTime.

      • NativeAffinityEdgeCaseTest, NativeAffinityErrorHandlingTest, NativeAffinityLibraryLoadingTest, NativeAffinityVersionTest, and NativeAffinityVersionIntegrationTest cover:

        • Library load state, version reporting, JNI error handling, argument bounds, concurrency, and safe repeated calls.
    • Updated existing tests:

      • AffinityThreadFactoryMain and AffinityThreadFactoryTest now use ExecutorService#execute with proper interruption handling.
      • LinuxJNAAffinityTest method naming aligned with standard Java test naming (linuxJna).
      • LockCheckTest and MultiProcessAffinityTest updated to use explicit UTF-8 encoding for lock files and avoid unnecessary locals.
      • BaseAffinityTest now uses a final TemporaryFolder rule to match JUnit best practices.
    • Added an OSGi test package net.openhft.affinity.osgi with package-info.java describing its role in verifying bundle packaging and service exposure.

  • Docs and guidance

    • Introduced root-level AGENTS.md:

      • Defines British English plus ISO-8859-1 policy for all content, including tools to detect non-compliant characters.
      • Documents high-value Javadoc rules (behavioural contracts only, no trivial “gets/sets” comments).
      • Captures CI expectations (mvn -q verify), PR etiquette, security checklist, Nine-Box requirement taxonomy, ADR format, and AsciiDoc conventions.
    • Added affinity/AGENTS.md to inherit repository rules and add module-specific guidelines (affinity purpose, build commands, quality gates, and platform guardrails).

    • Added CLAUDE.md with:

      • Project overview, module structure, build commands, architecture notes, and key classes for Claude Code.
      • Chronicle-wide conventions (language, charset, Javadoc, commits, test expectations, docs-sync behaviour).
  • Architecture and requirements docs

    • Added affinity/src/main/docs/decision-log.adoc capturing key architectural choices:

      • JNA as the primary native interface with JNI reserved for specialised paths.
      • File-based inter-process locking for CPU reservations.
      • /proc/cpuinfo-based topology parsing.
      • Recommended isolcpus usage for jitter reduction.
    • Added affinity/src/main/docs/project-requirements.adoc:

      • Reframes and tags functional and non-functional requirements with Nine-Box IDs (AFF-FN-*, AFF-NF-P-*, AFF-OPS-*, AFF-TEST-*, AFF-DOC-*, etc.).
      • Makes platform support, operability, UX, testing, risk, and documentation obligations explicit.
  • Miscellaneous functional tweaks

    • FileLockBasedLockChecker.getInstance() now returns a new instance to improve test isolation.
    • AffinityThreadFactoryMain and AffinityThreadFactoryTest slightly adjusted execution patterns but preserve behaviour.
    • LockChecker/LockReference formatting cleaned up while preserving API.

Non-functional / quality changes

  • Quality profiles and static analysis

    • Introduced a quality Maven profile in:

      • Root pom.xml.
      • affinity/pom.xml.
      • affinity-test/pom.xml.
    • Each quality profile:

      • Runs Checkstyle (3.6.0) with chronicle-quality-rules and a pinned Checkstyle core (10.26.1), including tests.
      • Runs SpotBugs (4.9.8.1) with chronicle-quality-rules include/exclude filters, effort=Max, threshold=Low, and includeTests=true, bound to process-test-classes.
    • Documented in TODO.md that affinity and affinity-test are now Checkstyle/SpotBugs-clean on Java 21 under the shared Chronicle configuration.

  • BOM and dependency alignment

    • affinity-test now imports net.openhft:third-party-bom:3.27ea7.
    • affinity module continues to track 3.27ea2 (unchanged in this diff) but now passes VERSION down to the native build.
  • Docs front-matter and AsciiDoc consistency

    • Added common header attributes to:

      • LICENSE.adoc and README.adoc: :toc:, :lang: en-GB, :source-highlighter: rouge.
    • Marked code sections in README.adoc with opts=novalidate where appropriate to avoid spurious AsciiDoc validation warnings.

    • Standardised AsciiDoc formatting guidance in AGENTS.md and ensured the new docs follow the same rules (automatic section numbering, no manual prefixes, correct list indentation).

  • Repository TODO and compliance traceability

    • Introduced a repository-scoped TODO.md that:

      • Links this module to the global ARCH_TODO.md, TODO_INDEX.md, ADOC_TODO.md, and ISO 9001 / ISO 27001 checklists.
      • Tracks open work on architecture overviews, decision logs, requirements, security reviews, performance targets, glossary terms, and test strategy.
      • Captures the current Checkstyle/SpotBugs status and notes deferred work via TODO_STATUS.md.
  • Security and robustness improvements

    • Native code:

      • Added defensive checks for unsupported platforms, error paths, and buffer bounds.
      • Ensured JNI functions ignore unused parameters explicitly and use safe string formatting.
    • Java:

      • Centralised NullAffinity fallback when JNA or native paths are not available.
      • Improved error messages around temp directory creation and lock-file content parsing.
    • AGENTS.md now includes a security checklist reminding reviewers to assess validation, authorisation, resource exhaustion, timing, and secrets handling on every PR.


Notes for reviewers

  • Behavioural scope

    • Affinity semantics are preserved by design: JNA remains the default for affinity control, JNI remains opt-in via NativeAffinity, and NullAffinity is the safety net when native support is missing.

    • The main behaviour changes are:

      • More predictable native library loading and error handling.
      • Clearer and more robust JNA/JNI interplay (including tests that validate JNA vs JNI parity on Linux).
  • Risk / impact

    • Risk is primarily around native integration on Linux/macOS; the new tests and error-handling paths are intended to reduce this by:

      • Failing fast with clear Java exceptions instead of C++ runtime errors.
      • Providing a well-defined fallback to JNA and NullAffinity if anything goes wrong in initialisation.
@peter-lawrey peter-lawrey changed the title Adv/develop Nov 21, 2025
peter-lawrey and others added 17 commits November 24, 2025 12:26
Security enhancements:
- Enable strict compiler flags (-Werror, -Wextra, -Wconversion)
- Add stack protection and buffer overflow detection
- Implement proper object file compilation with security flags

Version tracking:
- Embed PROJECT_VERSION from Maven into native library
- Add getVersion0() native method and VERSION constant
- Log native library version on load

Error handling improvements:
- Enhance exception safety in JNI layer
- Add proper unused parameter handling
- Fix platform-specific conditional compilation

Bug fixes:
- Fix unused function warning with conditional compilation
- Remove duplicate JNI_OnLoad definitions
- Fix syntax error in MacOSX.c
- Make MacOSX.c compilation conditional on Darwin platform

Testing:
- Add 27 comprehensive tests for new functionality
- Test version tracking, library loading, and error handling
- Verify thread safety and memory safety
- Total test count increased from 70 to 97 tests

All tests pass (97/97), build verified with mvn clean verify.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@peter-lawrey peter-lawrey changed the title Add AI agent guidance and refine affinity utilities Nov 26, 2025
@tgd tgd removed their request for review April 28, 2026 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants