Deciding what to fetch

Throughout the decision making process we accumulate reasons to decline to fetch any blocks. This type is used to wrap intermediate and final results.

All the various reasons we can decide not to fetch blocks from a peer.

It is worth highlighting which of these reasons result from competition among upstream peers.

• FetchDeclineInFlightOtherPeer: decline this peer because all the unfetched blocks of its candidate chain have already been requested from other peers. This reason reflects the least-consequential competition among peers: the competition that determines merely which upstream peer to burden with the request (eg the one with the best DeltaQ metrics). The consequences are relatively minor because the unfetched blocks on this peer's candidate chain will be requested regardless; it's merely a question of "From who?". (One exception: if an adversarial peer wins this competition such that the blocks are only requested from them, then it may be possible that this decision determines whether the blocks are ever received. But that depends on details of timeouts, a longer competing chain being soon received within those timeouts, and so on.)
• FetchDeclineChainNotPlausible: decline this peer because the node has already fetched, validated, and selected a chain better than its candidate chain from other peers (or from the node's own block forge). Because the node's current selection is influenced by what blocks other peers have recently served (or it recently minted), this reason reflects that peers indirectly compete by serving as long of a chain as possible and as promptly as possible. When the tips of the peers' selections are all within their respective forecast horizons (see ledgerViewForecastAt), then the length of their candidate chains will typically be the length of their selections, since the ChainSync is free to race ahead (in contrast, the BlockFetch pipeline depth is bounded such that it will, for a syncing node, not be able to request all blocks between the selection and the end of the forecast window). But if one or more of their tips is beyond the horizon, then the relative length of the candidate chains is more complicated, influenced by both the relative density of the chains' suffixes and the relative age of the chains' intersection with the node's selection (since each peer's forecast horizon is a fixed number of slots after the candidate's successor of that intersection).
• FetchDeclineConcurrencyLimit: decline this peer while the node has already fully allocated the artificially scarce maxConcurrentFetchPeers resource amongst its other peers. This reason reflects the least-fundamental competition: it's the only way a node would decline a candidate chain C that it would immediately switch to if C had somehow already been fetched (and any better current candidates hadn't). It is possible that this peer's candidate fragment is better than the candidate fragments of other peers, but that should only happen ephemerally (eg for a brief while immediately after first connecting to this peer).
• FetchDeclineChainIntersectionTooDeep: decline this peer because the node's selection has more than K blocks that are not on this peer's candidate chain. Typically, this reason occurs after the node has been declined---ie lost the above competitions---for a long enough duration. This decision only arises if the BlockFetch decision logic wins a harmless race against the ChainSync client once the node's selection gets longer, since ForkTooDeep disconnects from such a peer.

A chain suffix, obtained by intersecting a candidate chain with the current chain.

The anchor point of a ChainSuffix will be a point within the bounds of the current chain (withinFragmentBounds), indicating that it forks off in the last K blocks.

A ChainSuffix must be non-empty, as an empty suffix, i.e. the candidate chain is equal to the current chain, would not be a plausible candidate.

The combination of a ChainSuffix and a list of discontiguous AnchoredFragments:

• When comparing two CandidateFragments as candidate chains, we use the ChainSuffix.
• To track which blocks of that candidate still have to be downloaded, we use a list of discontiguous AnchoredFragments.

Keep only those candidate chains that are preferred over the current chain. Typically, this means that their length is longer than the length of the current chain.

Find the fragments of the chain suffix that we still need to fetch, these are the fragments covering blocks that have not yet been fetched and are not currently in the process of being fetched from this peer.

Typically this is a single fragment forming a suffix of the chain, but in the general case we can get a bunch of discontiguous chain fragments.

Filter a fragment. This is an optimised variant that will behave the same as filter if the following precondition is satisfied:

PRECONDITION: for all hdr in the chain fragment: if blockSlot hdr > maxSlotNo then the predicate should not hold for any header after hdr in the chain fragment.

For example, when filtering out already downloaded blocks from the fragment, it does not make sense to keep filtering after having encountered the highest slot number the ChainDB has seen so far: blocks with a greater slot number cannot have been downloaded yet. When the candidate fragments get far ahead of the current chain, e.g., 2k headers, this optimisation avoids the linear cost of filtering these headers when we know in advance they will all remain in the final fragment. In case the given slot number is NoSlotNo, no filtering takes place, as there should be no matches because we haven't downloaded any blocks yet.

For example, when filtering out blocks already in-flight for the given peer, the given maxSlotNo can correspond to the block with the highest slot number that so far has been in-flight for the given peer. When no blocks have been in-flight yet, maxSlotNo can be NoSlotNo, in which case no filtering needs to take place, which makes sense, as there are no blocks to filter out. Note that this is conservative: if a block is for some reason multiple times in-flight (maybe it has to be redownloaded) and the block's slot number matches the maxSlotNo, it will now be filtered (while the filtering might previously have stopped before encountering the block in question). This is fine, as the filter will now include the block, because according to the filtering predicate, the block is not in-flight.

A penultimate step of filtering, but this time across peers, rather than individually for each peer. If we're following the parallel fetch mode then we filter out blocks that are already in-flight with other peers.

Note that this does not cover blocks that are proposed to be fetched in this round of decisions. That step is covered in fetchRequestDecisions.