feat(rpc): serve states/{id}/validators/{validator_id} and v2 proposer duties, for Lighthouse's validator client

[pablodeymo]

🗒️ Description / Motivation

Lighthouse's validator client (v8.2.3) can't run on ethlambda beacon, because it needs two endpoints this node doesn't serve:

Endpoint	Before	What Lighthouse's VC does on the 404
GET /eth/v1/beacon/states/{state_id}/validators/{validator_id}	404 (only ?id= and POST served)	It resolves each key's index through this route. Without it, every validator is "inactive", so no attester, sync, liveness or proposer duty is ever requested
GET /eth/v2/validator/duties/proposer/{epoch}	404 (v1 served)	It asks for v2 only, with no fallback to v1, so "Failed to download proposer duties" every epoch and it never proposes

The first one blocks the end-to-end check #642 and #643 list as still to do: Lighthouse's validator client with --enable-doppelganger-protection. Its doppelganger service only calls liveness for validators it has indices for.

Found on a kurtosis devnet with Lighthouse's validator client talking only to ethlambda beacon. The VC logged Validator without index for each of its 64 keys and All validators inactive every slot.

What Changed

• crates/net/rpc/src/beacon/states.rs: GET /eth/v1/beacon/states/{state_id}/validators/{validator_id}. It returns one registry entry as an object, by index or by pubkey: the same entry the list endpoint returns, without the array.
• crates/net/rpc/src/beacon/validator.rs: GET /eth/v2/validator/duties/proposer/{epoch}. These are v1's duties with v2's dependent_root. v1 and v2 share one handler, parameterized by the dependent-root rule.
• docs/rpc.md: both routes, and the v2 dependent_root rule.
• tooling/kurtosis-validator/:
  • ethlambda_validator.client: lighthouse runs lighthouse vc in place of ethlambda's validator client, on the same derived keys and against the same beacon node list.
    • Lighthouse writes its definitions file into the validators directory, so it gets a writable copy of the keys.
    • doppelganger: true adds --enable-doppelganger-protection.
    • lighthouse_extra_params passes any other flags through.
  • network_params_lighthouse_vc.yaml runs it against ethlambda beacon alone, with doppelganger protection on.
• bin/ethlambda/src/main.rs: cargo fmt. A merge of beacon-chain-integration left the module list out of order, which is what fails Lint on feat(rpc): serve states/{id}/fork, config/deposit_contract and validator/duties/sync #642 and feat(rpc): serve validator/liveness #643.

Correctness / Behavior Guarantees

• validators/{validator_id}:
  • 404 for a validator the state doesn't hold, 400 for a malformed id.
  • state_id resolves the same way as on every other state endpoint.
• v2 proposer duties:
  • dependent_root is the block root at the slot before the previous epoch's first slot, since fulu fixes proposers an epoch ahead. Where that would underflow, it's the genesis block root.
  • 503 while syncing.
  • v1 is unchanged.

Tests Added / Run

• validators/{validator_id}:

  • one_validator_by_pubkey_or_index_is_the_list_entry
  • one_unknown_validator_is_a_404
  • one_malformed_validator_id_is_a_400

• v2 proposer duties:

  • v2_proposer_duties_are_v1_s
  • the_v2_dependent_root_is_the_block_before_the_previous_epoch
  • the_v2_dependent_root_is_genesis_where_it_would_underflow
  • v2_proposer_duties_answer_503_while_syncing

• cargo test -p ethlambda-rpc --profile release-fast: 136 passed, 0 failed.

• Kurtosis devnet, Lighthouse v8.2.3's validator client talking only to ethlambda beacon (built from this branch). The setup:

  • 2× geth+lighthouse hold validators [0, 112); Lighthouse's VC holds [112, 128) with --enable-doppelganger-protection. With network_params_lighthouse_vc.yaml's split, where the VC holds every key, the chain stops during the doppelganger wait (see below).
  • Lighthouse skips doppelganger for validators registered at genesis, so the VC was restarted at slot 40 to exercise it.

  Check	Result
  Index resolution (validators/{validator_id})	✅ all 16 resolved; before this PR: Validator without index for every key
  v2 proposer duties	✅ current_epoch_proposers: 7; it proposed slots 3, 4, 29 and 30. dependent_root matches Lighthouse's BN for epochs 0 and 1
  Attestations and aggregates through ethlambda	✅ published
  duties/sync (feat(rpc): serve states/{id}/fork, config/deposit_contract and validator/duties/sync #642)	✅ Validator in sync committee for 112 and 114
  Doppelganger via liveness (feat(rpc): serve validator/liveness #643)	✅ after the restart: Listening for doppelgangers (16), then Found no doppelganger … further_checks_remaining: 0 and Doppelganger detection complete: starting validator. The validators unmuted at slot 128 (epoch 4) and attested at once. liveness answered false for the muted keys and true for a participant's
  Finality	✅ justified 3, finalized 2 at epoch 4

• Found on the devnet, not fixed here:

  • Sync-committee endpoints missing. The node doesn't serve sync_committee_subscriptions, pool/sync_committees or the contribution endpoints. Lighthouse logs Unable to publish sync committee messages and No aggregate contribution found (CRIT) every slot. This is the known gap feat(rpc): serve states/{id}/fork, config/deposit_contract and validator/duties/sync #642 describes.
  • Duties refused at each epoch boundary. duties/attester and duties/proposer bound the requested epoch by the head state's epoch, not the clock's (validator.rs, from the validator API work). At an epoch's first slot, before its block arrives, the next epoch's duties are a 400 (epoch is not within one epoch of the head state's). Lighthouse retries next slot and gets them, but it logs an ERROR every epoch. Lighthouse's BN advances the state instead; its own VCs logged no such failure.
  • Stall with the committed split. network_params_lighthouse_vc.yaml's split, where the VC holds all 128 keys, risks a stall: while doppelganger mutes the VC's validators, a run of more than 8 missed slots puts Lighthouse past its sync tolerance and every VC stops. An earlier run with a 64/64 split stopped at slot 55.
  • Proposer refused at the slot boundary. Lighthouse's VC asked to propose slot 29 a second time 3 ms before slot 30, once the block was already imported; ethlambda answered 400 slot is not after the head block. Harmless: slashing protection would refuse the second signature too.

Related Issues / PRs

• Stacked on feat(rpc): serve validator/liveness #643, which is stacked on feat(rpc): serve states/{id}/fork, config/deposit_contract and validator/duties/sync #642. Retarget to beacon-chain-integration once those merge.
• Unblocks the Lighthouse end-to-end check listed under "Not yet run" in feat(rpc): serve states/{id}/fork, config/deposit_contract and validator/duties/sync #642 and feat(rpc): serve validator/liveness #643.

✅ Verification Checklist

• Ran make fmt — clean
• Ran make lint (clippy with -D warnings) — clean
• Ran make test (test-consensus plus test-node, at release-fast) — only ethlambda-rpc locally (136 passed), the only crate with code changes; the full suite runs in CI

[github-actions]

🤖 Kimi Code Review

I'll review this PR which adds Beacon API endpoints for validator lookup and v2 proposer duties, plus Lighthouse VC integration for testing.

crates/net/rpc/src/beacon/states.rs

Line 370-371: validator_response — Potential panic on as ValidatorIndex cast.

.ok_or(ApiError::NotFound("validator not found"))?
as ValidatorIndex,

If position() returns usize::MAX (or any value > u64::MAX), this truncates silently. While practically impossible with valid beacon states (validator count is bounded), this should use try_into() for correctness:

.ok_or(ApiError::NotFound("validator not found"))?
.try_into()
.map_err(|_| ApiError::Internal("validator index overflow"))?,

Line 373-374: state.validator(index) returns Result but error is mapped to NotFound. This is correct behavior per spec, but the comment should note that index bounds checking happens here.

Line 376-379: Balance lookup inconsistency handling — Good. Using ApiError::Internal for registry/balance length mismatch is appropriate (this is an invariant violation).

Line 381: status.name() — Ensure this returns the exact string the Beacon API spec requires (e.g., "active_ongoing", not "ActiveOngoing"). Not visible in diff, but worth verifying.

crates/net/rpc/src/beacon/validator.rs

Line 400-416: get_proposer_duties_v2 — The 503 while syncing is only implemented for v2, not v1. This is intentional per PR description, but consider: should v1 also return 503 when syncing? The spec says "while syncing" for proposer duties generally. However, since this matches the documented behavior change, it's acceptable if documented.

Line 418-439: DependentRoot — Good abstraction. The saturating_sub pattern correctly handles genesis underflow.

Line 431: compute_start_slot_at_epoch(epoch).saturating_sub(1) — This duplicates logic in block_root_at_or_before which previously did first_slot.saturating_sub(1). The refactor is cleaner.

Line 485: block_root_at_or_before(&state, head_root, dependent.slot(epoch)) — Correctly replaces the inline computation.

Line 758-762: Test get_v2 helper — Default::default() for SyncStatusController is used. Ensure this derives Default and produces SyncStatus::Synced or similar. The test at line 812 creates explicit SyncStatus::Syncing, so the default must be non-syncing.

Line 784-785: fulu_state() fixture — Test assumes this produces state at epoch 1. The assertion at line 802 verifies this, but if the fixture changes, tests break. Consider making the epoch explicit in the fixture name or parameterizing.

Line 791: compute_start_slot_at_epoch(next - 1) - 1 — This is raw subtraction, not saturating_sub. In the test context with next = 2, this is safe, but inconsistent with production code style. Use saturating_sub for consistency.

Line 802-803: The assertion assert_eq!(compute_epoch_at_slot(state.slot()), 1, "the fixture's epoch") is a test precondition check. Good practice.

tooling/kurtosis-validator/main.star

Line 123: if vc.get("client", "ethlambda") == "lighthouse": — The default string "ethlambda" should be a constant, matching the ethlambda_validator key structure.

Line 170-220: launch_lighthouse_vc — The cp -r {}/raw /data/ copies keys every startup. For large validator sets this adds startup latency. Acceptable for test tooling.

Line 198: --init-slashing-protection — This wipes slashing protection on restart. Fine for ephemeral testnets, but dangerous if this code is copied for production. Add a prominent comment warning.

Line 204: vc.get("lighthouse_extra_params", []) — No validation of extra params. Acceptable for test tooling.

Line 211: entrypoint=["sh", "-c"] with cmd as single string — This is a shell injection risk if flags contains user input. In test tooling context with controlled configs, acceptable, but consider using array form:

cmd=["sh", "-c", "mkdir -p /data && cp -r {}/raw /data/ && lighthouse vc {}".format(KEYS_MOUNT, " ".join(flags))],

Actually, the current code already does this correctly — the list contains one string. No issue.

Line 214: vc.get("lighthouse_image", "sigp/lighthouse:latest") — Using :latest tag is non-reproducible. Consider pinning to a specific version for CI stability. The YAML uses sigp/lighthouse:latest implicitly via default.

tooling/kurtosis-validator/network_params_lighthouse_vc.yaml

Line 39: image: ghcr.io/lambdaclass/ethlambda:beacon-api-lighthouse-defaults — This tag appears to be PR-specific. Ensure this is updated or parameterized before merge, or document as intentional for testing.

General Observations

1. No XMSS or consensus-critical changes — This PR is purely RPC/API layer and test infrastructure. No fork choice, attestation, or finalization logic is modified.

2. SSZ not involved — All new endpoints return JSON per Beacon API spec. No SSZ encoding concerns.

3. Error handling pattern — Consistent use of ApiError with ? operator and .into_response(). Good.

4. Test coverage — Excellent test coverage for:

   • Happy path (pubkey and index lookup)
   • 404 cases (unknown validator)
   • 400 cases (malformed input)
   • v1/v2 duty equivalence
   • v2 dependent root correctness
   • Genesis underflow handling
   • Syncing 503 response

Summary

The PR is well-structured and correct. Key points:

• Minor: Fix as ValidatorIndex cast in states.rs:370 with try_into()
• Minor: Use saturating_sub in test at validator.rs:791 for consistency
• Suggestion: Pin Lighthouse image version in YAML or document :latest choice
• Good: Clean abstraction of DependentRoot, proper 503 handling, comprehensive tests

---

Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

[github-actions]

🤖 Claude Code Review

Review of PR 653

I couldn't review the actual change. The checkout is a single squashed commit (ea90354) with no parent, so there is no diff against main to inspect. I also didn't run cargo fmt --check: the tool refused the compound command and I didn't retry it on its own.

What I did check is the one change the commit message describes. The mod declarations at bin/ethlambda/src/main.rs:1-11 are in alphabetical order (banner, beacon, benchmark, checkpoint_sync, cli, command, fd_limit, keygen, network, validator, version). That matches what the commit says it fixes, so the Lint failure on module order should be resolved.

• Correctness and security: nothing to assess for a pure reordering of mod lines, which has no behavioral effect.
• Message wording: the commit message is long and describes the cause rather than the change. A short subject such as "Run cargo fmt on main.rs" would read better in history.
• Scope: if the PR also carries the full beacon-chain-integration merge (STF, p2p, rpc, types), it needs a review against main. That is outside what I could examine here.

---

Automated review by Claude (Anthropic) · sonnet · custom prompt

[github-actions]

🤖 Codex Code Review

Found two review items; the new Beacon API route itself looks solid otherwise.

• Shell injection in Kurtosis launcher: tooling/kurtosis-validator/main.star:210 builds sh -c from unescaped user-controlled values (graffiti, suggested_fee_recipient, lighthouse_extra_params, and potentially beacon node URLs via config flow). Any quote/command separator in those fields can alter the container command. For infra/test tooling this is still worth fixing: either invoke lighthouse directly via argv-style cmd, or shell-escape each interpolated flag before joining.

• Service naming inconsistency may break expected labeling/ops flow: tooling/kurtosis-validator/main.star:205 hardcodes the service name to vc-lighthouse, while the rest of the script treats the validator as logically named by vc.get("name", "ethlambda-vc") (tooling/kurtosis-validator/main.star:116, tooling/kurtosis-validator/main.star:118). If users run multiple instances or expect the configured name to carry through operationally, this becomes surprising and brittle. Consider using the configured name for the service as well.

• Beacon API endpoint correctness: crates/net/rpc/src/beacon/states.rs:370–crates/net/rpc/src/beacon/states.rs:405 is consistent with the existing /validators implementation: same status derivation, same execution_optimistic/finalized fields, 400 on malformed IDs, 404 on missing validator, and no unsafe memory/error-handling patterns.

• Performance note, not a blocker: crates/net/rpc/src/beacon/states.rs:379–crates/net/rpc/src/beacon/states.rs:384 does a linear scan for pubkey lookup. That is probably fine for this single-validator endpoint and matches the code comment, but it is still O(n) over the registry. If this path becomes hot on large validator sets, a temporary pubkey→index map or store-level index would help.

No fork-choice / justification / state-transition / SSZ / signature-path concerns are introduced by this PR; the consensus-critical surface here appears unchanged.

---

Automated review by OpenAI Codex · gpt-5.4 · custom prompt

[MegaRedHand]

Comments below. The v2 dependent_root rule and the head-state bound on v2 are the two that affect Lighthouse's validator client; the kurtosis config one decides whether the liveness check actually runs.

[MegaRedHand]

Medium: the requested epoch is bounded by the head state's epoch (the offset <= MIN_SEED_LOOKAHEAD check below), not the clock. Lighthouse's VC asks v2 for current_epoch and current_epoch + 1 on every poll (poll_beacon_proposers in duties_service.rs). From the start of epoch E until a block lands in E, the head state is still in E-1, so the E+1 request is a 400 and Lighthouse logs an error each poll; with several missed slots at the start of E, next-epoch duties stay unavailable that long.

Advancing the head state to the clock's epoch (as attestation_data does through fork_choice::checkpoint_state) would fix it. v1 has the same bound, so it isn't new, but v2 is the path Lighthouse uses.

[MegaRedHand]

Minor (one epoch per network): v2 always uses the post-Fulu rule. Lighthouse's proposer_shuffling_decision_slot (consensus/types/src/core/chain_spec.rs) is fork-aware: it uses start(epoch - 1) - 1 only when Fulu is active at epoch - 1, and the v1 rule otherwise. At the Fulu fork epoch F itself, the proposers were fixed by the upgrade from the end of epoch F-1, so v2 there equals v1.

With fulu_fork_epoch: 0 (the kurtosis configs) the two agree. With a later fork epoch, at epoch F this answers a different root than Lighthouse's BN, so a VC failing over between the two sees a spurious reorg, and a reorg of F-1's tail goes unnoticed. Checking the fork at epoch - 1, and using MIN_SEED_LOOKAHEAD instead of the literal 1, would match.

(This rule is fork-dependent and attester_duties' identical-looking formula is not, so they shouldn't be merged into one helper.)

[MegaRedHand]

Minor: v2 is gated on sync status but v1 isn't, although both answer from the same head-state lookahead. If the gate matters, v1 (what ethlambda validator uses) needs it too; if it doesn't, v2 can drop it. Also, this if Syncing { 503 } block is now in three handlers (sync duties, liveness, v2 proposer duties); a reject_if_syncing helper or a route_layer on the gated routes would keep them identical.

[MegaRedHand]

Medium (tooling): as committed, this config can't exercise liveness. Lighthouse turns doppelganger protection off for validators registered in the genesis epoch (doppelganger_service/src/lib.rs: remaining_epochs = 0 when current_epoch <= genesis_epoch), and the VC starts with the network. Restarting the VC later to trigger it mutes all 128 validators, because the participants run validator_count: 0, so the chain stops. A split where the CL participants keep most of the keys and the Lighthouse VC holds a subset that it starts (or restarts) after epoch 0 would test it.

[MegaRedHand]

Nit: seats are drawn with replacement, so each of 128 validators has about a 1.8% chance ((127/128)^512) of holding none; typically one or two validators get no sync duty. Someone checking "every key gets a sync duty" against this would suspect the endpoint.

[MegaRedHand]

Nit: sigp/lighthouse:latest floats. Pinning the version this was checked with (v8.2.3) keeps a future Lighthouse release from changing what this config tests without a change here.

[MegaRedHand]

Nit: get_one was inserted under the existing doc comment, so "What ethlambda validator sends: its keys, to learn their indices." now documents the GET helper (which is what Lighthouse uses), and a_pubkey_resolves_to_its_index lost its comment.

[MegaRedHand]

Nit (perf): each lookup by pubkey is a linear scan of the registry, run on a tokio worker. Lighthouse's VC calls this once per key at startup and again for keys it has no index for, so on a mainnet-sized registry that's one full scan per key. A pubkey-to-index map (or spawn_blocking) would help if this runs on a large network; fine for kurtosis.

[MegaRedHand]

Nit: the entry building here (id match, ValidatorStatus::of, balance lookup, ValidatorEntry) repeats validators_response. Running the list selection with one id and returning its single match or a 404 would keep one copy.

[MegaRedHand]

Nit: the module docs still say this file serves the endpoints under /eth/v1/validator/; it now serves /eth/v2/validator/duties/proposer too. Same for the header list in states.rs, which omits /fork (from #642) and /validators/{validator_id}.