User-Service-Router API Reference

Version: 1.0 Date: 2025-11-15 Canister: user-service-router Status: Planned (Phase 2) Candid Interface: Same as user_service.did

Overview

The user-service-router canister provides transparent request routing and shard coordination for horizontal scaling of user-service in the Hello World DAO ecosystem. It exposes an identical API to the standalone user-service canister, enabling zero-downtime migration from standalone to multi-shard deployments.

Purpose

The router acts as a reverse proxy that:

Routes Requests: Directs API calls to appropriate shard based on email hash
Load Distribution: Balances user data evenly across multiple shards
Backward Compatibility: Provides identical API to standalone user-service
Seamless Scaling: Supports adding/removing shards without client changes
Query Aggregation: Combines results from multiple shards for broadcast queries

Key Features

Hash-based deterministic routing for consistent shard assignment
Transparent proxying of all user-service methods
Broadcast queries for principal-based lookups
Result aggregation for statistics and pagination
Shard health monitoring and metrics collection
Controller-only shard configuration management

Deployment Model

The router enables two distinct deployment topologies:

Standalone Deployment (Phase 1, Current):

Single user-service canister handles all operations
Direct API access, no routing overhead
Capacity: ~500,000 users

Router + Shards Deployment (Phase 2, Future):

Router canister distributes requests across N shard canisters
Linear scalability with shard count
Capacity: Millions of users (3 shards = ~1.5M users)

Architecture

Request Flow

mermaid

sequenceDiagram
    participant Client
    participant Router as user-service-router
    participant Shard1 as user-service Shard 1
    participant Shard2 as user-service Shard 2
    participant Shard3 as user-service Shard 3

    Note over Client,Shard3: Individual Registration (Email-Based Routing)
    Client->>Router: submit_individual(request, address)
    Note over Router: Calculate: hash(email_hash) % 3 = 1
    Router->>Shard2: submit_individual_shard(request, address)
    Shard2->>Shard2: Validate, Store, Generate Code
    Shard2-->>Router: IndividualResult
    Router-->>Client: IndividualResult

    Note over Client,Shard3: Principal Lookup (Broadcast Query)
    Client->>Router: get_user_by_principal(principal)
    Note over Router: Broadcast to all shards
    par Query all shards
        Router->>Shard1: get_user_by_principal_shard(principal)
        Router->>Shard2: get_user_by_principal_shard(principal)
        Router->>Shard3: get_user_by_principal_shard(principal)
    end
    Shard1-->>Router: None
    Shard2-->>Router: Some(IndividualRecord)
    Shard3-->>Router: None
    Note over Router: Return first match
    Router-->>Client: Some(IndividualRecord)

    Note over Client,Shard3: Statistics Aggregation (Reduce)
    Client->>Router: get_stats()
    par Query all shards
        Router->>Shard1: get_stats_shard()
        Router->>Shard2: get_stats_shard()
        Router->>Shard3: get_stats_shard()
    end
    Shard1-->>Router: Stats { total: 500K, verified: 450K }
    Shard2-->>Router: Stats { total: 500K, verified: 480K }
    Shard3-->>Router: Stats { total: 500K, verified: 470K }
    Note over Router: Aggregate: sum all fields
    Router-->>Client: Stats { total: 1.5M, verified: 1.4M }

Routing Architecture

┌─────────────────┐
│ Client Apps     │
│ (Web/Mobile)    │
└────────┬────────┘
         │ Candid API (identical to user-service)
         ▼
┌─────────────────────────────────────────┐
│  user-service-router                    │
│                                         │
│  ┌────────────────────────────────┐    │
│  │ Hash-Based Routing Engine      │    │
│  │ - Calculate: hash % shard_count│    │
│  │ - Deterministic assignment     │    │
│  └────────────────────────────────┘    │
│                                         │
│  ┌────────────────────────────────┐    │
│  │ Query Aggregation              │    │
│  │ - Broadcast queries            │    │
│  │ - Result merging               │    │
│  └────────────────────────────────┘    │
│                                         │
│  ┌────────────────────────────────┐    │
│  │ Shard Management               │    │
│  │ - Configuration                │    │
│  │ - Health monitoring            │    │
│  └────────────────────────────────┘    │
└──────┬───────────┬──────────┬──────────┘
       │           │          │
       ▼           ▼          ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Shard 1  │ │ Shard 2  │ │ Shard 3  │
│ email %  │ │ email %  │ │ email %  │
│ 3 == 0   │ │ 3 == 1   │ │ 3 == 2   │
│          │ │          │ │          │
│ 500K     │ │ 500K     │ │ 500K     │
│ users    │ │ users    │ │ users    │
└──────────┘ └──────────┘ └──────────┘

Routing Behavior

The router implements three distinct routing strategies based on operation type:

Strategy 1: Email-Based Routing

Operations: Individual registration, verification, deletion

Routing Key: email_hash field from request

Algorithm:

rust

fn route_by_email_hash(email_hash: &str) -> Principal {
    let hash_value = calculate_hash(email_hash);
    let shard_index = (hash_value % shard_count) as usize;
    SHARDS[shard_index]
}

Methods Using This Strategy:

submit_individual(request, address) - Routes by request.email_hash
verify_code(individual_id, code) - Extracts hash from individual_id prefix
resend_verification_code(email) - Hashes email parameter

Properties:

Deterministic: Same email always routes to same shard
Balanced: Hash distribution ensures even load
Efficient: O(1) lookup time

Strategy 2: Individual-Based Routing

Operations: Address management, individual updates

Routing Key: individual_id parameter (contains shard identifier in prefix)

Algorithm:

rust

fn route_by_individual_id(individual_id: &str) -> Principal {
    // individual_id format: "ind_{timestamp}{random}"
    // Timestamp encodes original email_hash routing decision
    let email_hash = extract_email_hash_from_id(individual_id);
    route_by_email_hash(&email_hash)
}

Methods Using This Strategy:

add_address(individual_id, address) - Routes by individual_id
update_address(address_id, updates) - Looks up address to get individual_id, then routes
delete_address(address_id) - Looks up address to get individual_id, then routes
set_primary_address(individual_id, address_id) - Routes by individual_id
get_addresses_for_individual(individual_id) (query) - Routes by individual_id
get_primary_address(individual_id) (query) - Routes by individual_id

Lookup Caching: For address_id-based operations, the router maintains a cache:

rust

ADDRESS_TO_SHARD_CACHE: RefCell<HashMap<String, Principal>>

Strategy 3: Broadcast Queries

Operations: Principal-based lookups

Behavior: Query all shards, return first match

Algorithm:

rust

async fn broadcast_query<T>(
    method: &str,
    args: Vec<u8>
) -> Option<T> {
    let futures: Vec<_> = SHARDS
        .iter()
        .map(|shard| call(*shard, method, args.clone()))
        .collect();

    let results = join_all(futures).await;
    results.into_iter().find_map(|r| r.ok().flatten())
}

Methods Using This Strategy:

get_user_by_principal(principal) - User could be on any shard
get_current_user() - Uses caller principal, could be on any shard

Performance: O(N) where N is shard count, queries run in parallel

Strategy 4: Aggregation Queries

Operations: Statistics, listing, pagination

Behavior: Query all shards, combine/aggregate results

Algorithm:

rust

async fn aggregate_stats() -> Stats {
    let futures: Vec<_> = SHARDS
        .iter()
        .map(|shard| call(*shard, "get_stats_shard", ()))
        .collect();

    let results = join_all(futures).await;
    let mut total = Stats::default();

    for stats in results {
        total.total_individuals += stats.total_individuals;
        total.verified_individuals += stats.verified_individuals;
        total.pending_verifications += stats.pending_verifications;
    }

    total
}

Methods Using This Strategy:

get_stats() - Sums totals across all shards
list_individuals() (controller only) - Concatenates results
list_individuals_paginated(page, page_size) - Distributed pagination

Performance: O(N) queries, results combined by router

Routing Overhead

Latency Impact

Operation Type	Standalone	Router (3 shards)	Overhead
Single-shard query	~100ms	~150ms	+50ms
Single-shard update	~2s	~2.5s	+500ms
Broadcast query	~100ms	~200ms	+100ms
Aggregated stats	~100ms	~250ms	+150ms

Overhead Sources:

Routing calculation: ~10ms
Inter-canister call setup: ~20-40ms
Result deserialization: ~10-20ms
Cache lookups: ~5-10ms

Throughput Scaling

Metric	Standalone	Router (3 shards)	Scaling Factor
Queries per second	1,000 qps	3,000 qps	3x
Updates per second	100 tps	300 tps	3x
Storage capacity	500K users	1.5M users	3x
Memory usage	4GB	12GB total	3x

Linear Scaling: Adding N shards multiplies capacity by N

User-Service API Methods

The router exposes the complete user-service API with transparent proxying. All methods have identical signatures to the standalone user-service.

Individual Management

submit_individual

Signature:

candid

submit_individual : (IndividualRequest, opt AddressRequest) -> (IndividualResult);

Routing: Email-based (by request.email_hash)

Access: Public

Parameters:

request: IndividualRequest - Encrypted individual data
address: opt AddressRequest - Optional initial address

Returns: IndividualResult

Routing Behavior:

Extracts email_hash from request
Calculates: shard_index = hash(email_hash) % shard_count
Routes to: SHARDS[shard_index].submit_individual_shard(request, address)
Returns result unchanged

Example:

javascript

const result = await userServiceRouter.submit_individual(
  {
    email_encrypted: btoa("user@example.com"),
    first_name_encrypted: btoa("John"),
    last_name_encrypted: btoa("Doe"),
    email_hash: sha256("user@example.com"),
    encryption_key_id: "key_001",
    encryption_type: { Temporary: null },
    email_plaintext_for_verification: "user@example.com"
  },
  [{
    address_type: { Home: null },
    country: "US",
    city: "San Francisco",
    postal_code: "94103",
    is_primary: true
  }]
);

Performance: +500ms overhead vs standalone

verify_code

Verify email using verification code.

Signature:

candid

verify_code : (text, text) -> (VerifyResult);

Routing: Individual-based (by individual_id)

Access: Public

Parameters:

individual_id: text - Individual identifier
code: text - 6-digit verification code

Returns: VerifyResult

Routing Behavior:

Extracts email hash from individual_id prefix
Routes to appropriate shard based on hash
Calls: shard.verify_code_shard(individual_id, code)

Example:

javascript

const result = await userServiceRouter.verify_code(
  "ind_1699999999999123456",
  "123456"
);

Performance: +500ms overhead vs standalone

get_current_user

Get authenticated user's individual record.

Signature:

candid

get_current_user : () -> (opt IndividualRecord) query;

Routing: Broadcast query to all shards

Access: Public

Returns: opt IndividualRecord

Routing Behavior:

Extracts caller principal via ic_cdk::caller()
Broadcasts query to all shards in parallel
Each shard searches for matching ii_principal
Returns first non-empty result

Example:

javascript

const user = await userServiceRouter.get_current_user();

if (user.length > 0) {
  console.log("User:", user[0].email);
}

Performance: +100ms overhead (parallel broadcast)

get_user_by_principal

Look up user by Internet Identity principal.

Signature:

candid

get_user_by_principal : (principal) -> (opt IndividualRecord) query;

Routing: Broadcast query to all shards

Access: Public

Parameters:

user_principal: principal - Principal to search for

Returns: opt IndividualRecord

Routing Behavior:

Broadcasts query to all shards in parallel
Each shard searches its partition
Returns first match found
Returns empty option if no matches

Example:

javascript

const principal = Principal.fromText("aaaaa-aa");
const user = await userServiceRouter.get_user_by_principal(principal);

Performance: +100ms overhead (parallel broadcast)

Note: User could be on any shard, so broadcast is required

Address Management

add_address

Add new address to individual.

Signature:

candid

add_address : (text, AddressRequest) -> (variant { Ok : text; Err : text });

Routing: Individual-based (by individual_id)

Access: Public

Parameters:

individual_id: text - Individual identifier
address_data: AddressRequest - Address details

Returns: Result<String, String> - Address ID on success

Routing Behavior:

Extracts email hash from individual_id
Routes to appropriate shard
Calls: shard.add_address_shard(individual_id, address_data)
Caches address_id -> shard mapping for future operations

Example:

javascript

const result = await userServiceRouter.add_address(
  "ind_1699999999999123456",
  {
    address_type: { Work: null },
    country: "US",
    state: ["NY"],
    city: "New York",
    postal_code: "10001",
    is_primary: false
  }
);

Performance: +500ms overhead vs standalone

update_address

Update existing address.

Signature:

candid

update_address : (text, AddressRequest) -> (variant { Ok : null; Err : text });

Routing: Address-based with cache lookup

Access: Public

Parameters:

address_id: text - Address identifier
address_data: AddressRequest - Updated address details

Returns: Result<(), String>

Routing Behavior:

Checks cache for address_id -> shard mapping
If cache miss, broadcasts query to find owning shard
Caches mapping for future requests
Routes to: shard.update_address_shard(address_id, address_data)

Example:

javascript

const result = await userServiceRouter.update_address(
  "addr_1699999999999789012",
  {
    address_type: { Work: null },
    country: "US",
    postal_code: "10002",
    is_primary: true
  }
);

Performance:

Cache hit: +500ms overhead
Cache miss: +600ms overhead (includes broadcast lookup)

delete_address

Delete address by ID.

Signature:

candid

delete_address : (text) -> (variant { Ok : null; Err : text });

Routing: Address-based with cache lookup

Access: Public

Parameters:

address_id: text - Address identifier

Returns: Result<(), String>

Routing Behavior:

Looks up shard from cache (or broadcasts if cache miss)
Routes to: shard.delete_address_shard(address_id)
Removes address_id from cache after successful deletion

Example:

javascript

const result = await userServiceRouter.delete_address(
  "addr_1699999999999789012"
);

Performance: +500ms overhead (assumes cache hit)

set_primary_address

Set specific address as primary for individual.

Signature:

candid

set_primary_address : (text, text) -> (variant { Ok : null; Err : text });

Routing: Individual-based (by individual_id)

Access: Public

Parameters:

individual_id: text - Individual identifier
address_id: text - Address to set as primary

Returns: Result<(), String>

Routing Behavior:

Routes by individual_id to appropriate shard
Calls: shard.set_primary_address_shard(individual_id, address_id)

Example:

javascript

const result = await userServiceRouter.set_primary_address(
  "ind_1699999999999123456",
  "addr_1699999999999789012"
);

Performance: +500ms overhead vs standalone

get_addresses_for_individual

Get all addresses for an individual.

Signature:

candid

get_addresses_for_individual : (text) -> (vec Address) query;

Routing: Individual-based (by individual_id)

Access: Public

Parameters:

individual_id: text - Individual identifier

Returns: vec Address

Routing Behavior:

Routes by individual_id to single shard
Calls: shard.get_addresses_for_individual_shard(individual_id)

Example:

javascript

const addresses = await userServiceRouter.get_addresses_for_individual(
  "ind_1699999999999123456"
);

Performance: +50ms overhead (query operation)

get_primary_address

Get primary address for an individual.

Signature:

candid

get_primary_address : (text) -> (opt Address) query;

Routing: Individual-based (by individual_id)

Access: Public

Parameters:

individual_id: text - Individual identifier

Returns: opt Address

Routing Behavior:

Routes by individual_id to single shard
Calls: shard.get_primary_address_shard(individual_id)

Example:

javascript

const primary = await userServiceRouter.get_primary_address(
  "ind_1699999999999123456"
);

Performance: +50ms overhead (query operation)

Statistics

get_stats

Get aggregated service statistics across all shards.

Signature:

candid

get_stats : () -> (Stats) query;

Routing: Aggregation query (all shards)

Access: Public

Returns: Stats

candid

type Stats = record {
  total_individuals : nat64;
  verified_individuals : nat64;
  pending_verifications : nat64;
};

Routing Behavior:

Queries all shards in parallel: shard.get_stats_shard()
Aggregates results by summing all fields:
- total = sum(shard.total_individuals)
- verified = sum(shard.verified_individuals)
- pending = sum(shard.pending_verifications)
Returns aggregated Stats record

Example:

javascript

const stats = await userServiceRouter.get_stats();

console.log(`Total users: ${stats.total_individuals}`);
console.log(`Verified: ${stats.verified_individuals}`);
console.log(`Pending: ${stats.pending_verifications}`);

Performance: +150ms overhead (parallel aggregation)

Aggregation Logic:

rust

async fn aggregate_stats() -> Stats {
    let mut total = Stats {
        total_individuals: 0,
        verified_individuals: 0,
        pending_verifications: 0,
    };

    let futures: Vec<_> = SHARDS
        .iter()
        .map(|shard| call(*shard, "get_stats_shard", ()))
        .collect();

    let results = join_all(futures).await;

    for stats in results {
        total.total_individuals += stats.total_individuals;
        total.verified_individuals += stats.verified_individuals;
        total.pending_verifications += stats.pending_verifications;
    }

    total
}

Controller-Only Methods

list_individuals

List all individuals across all shards (controller only).

Signature:

candid

list_individuals : () -> (variant { Ok : vec IndividualRecord; Err : text }) query;

Routing: Aggregation query (all shards)

Access: Controller only

Returns: Result<vec IndividualRecord, String>

Routing Behavior:

Verifies caller is controller
Queries all shards in parallel: shard.list_individuals_shard()
Concatenates all results into single vector
Returns combined list

Example:

javascript

const result = await userServiceRouter.list_individuals();

if ('Ok' in result) {
  console.log(`Total users: ${result.Ok.length}`);
}

Performance: +100ms overhead (parallel concatenation)

Note: Returns could be very large for high user counts

list_individuals_paginated

Paginated listing of individuals with distributed pagination (controller only).

Signature:

candid

list_individuals_paginated : (nat64, nat64) -> (variant { Ok : PaginatedResult; Err : text }) query;

Routing: Aggregation query with distributed pagination

Access: Controller only

Parameters:

page: nat64 - Zero-indexed page number
page_size: nat64 - Results per page

Returns: Result<PaginatedResult, String>

candid

type PaginatedResult = record {
  individuals : vec IndividualRecord;
  total : nat64;
  page : nat64;
  page_size : nat64;
  has_next : bool;
};

Routing Behavior (Complex):

Queries all shards for totals: shard.get_stats_shard()
Calculates global offset: offset = page * page_size
Determines which shards to query based on offset and page_size
Fetches appropriate ranges from each shard
Concatenates results
Calculates has_next flag

Example:

javascript

const result = await userServiceRouter.list_individuals_paginated(0, 100);

if ('Ok' in result) {
  const data = result.Ok;
  console.log(`Page ${data.page}: ${data.individuals.length} of ${data.total}`);
  console.log(`Has next: ${data.has_next}`);
}

Performance: +150ms overhead (distributed pagination calculation)

Pagination Algorithm:

rust

async fn paginated_aggregation(
    page: u64,
    page_size: u64
) -> Result<PaginatedResult, String> {
    // Step 1: Get total counts from all shards
    let shard_totals = get_shard_totals().await;
    let global_total: u64 = shard_totals.iter().sum();

    // Step 2: Calculate global offset
    let global_offset = page * page_size;

    // Step 3: Determine which shards to query
    let mut current_offset = 0;
    let mut results = Vec::new();
    let mut remaining = page_size;

    for (shard, shard_total) in SHARDS.iter().zip(shard_totals.iter()) {
        if current_offset + shard_total <= global_offset {
            // Skip this shard entirely
            current_offset += shard_total;
            continue;
        }

        if remaining == 0 {
            break;
        }

        // Calculate shard-local offset and limit
        let local_offset = if current_offset < global_offset {
            global_offset - current_offset
        } else {
            0
        };

        let local_limit = remaining.min(shard_total - local_offset);

        // Query this shard
        let shard_results = call(
            *shard,
            "list_individuals_paginated_shard",
            (local_offset / page_size, local_limit)
        ).await?;

        results.extend(shard_results.individuals);
        remaining -= shard_results.individuals.len() as u64;
        current_offset += shard_total;
    }

    Ok(PaginatedResult {
        individuals: results,
        total: global_total,
        page,
        page_size,
        has_next: global_offset + page_size < global_total,
    })
}

initialize_signing_key

Initialize canister signing key (controller only).

Signature:

candid

initialize_signing_key : () -> (variant { Ok : text; Err : text });

Routing: Not applicable (router-local operation)

Access: Controller only

Returns: Result<String, String>

Routing Behavior: This method is NOT proxied to shards. The router initializes its own signing key for router-specific operations (if needed).

Example:

javascript

const result = await userServiceRouter.initialize_signing_key();

Note: In sharded deployments, each shard must initialize its own signing key separately via direct canister calls.

Router-Specific Methods

These methods are unique to the router and do not exist in the standalone user-service.

Shard Configuration

configure_shards

Configure complete shard list (replaces existing configuration).

Signature:

candid

configure_shards : (vec principal) -> (variant { Ok : null; Err : text });

Access: Controller only

Parameters:

shards: vec principal - List of shard canister principals

Returns: Result<(), String>

Behavior:

Validates caller is controller
Validates all principals are valid canister IDs
Replaces entire shard list
Clears routing cache
Resets metrics

Example:

bash

dfx canister call user-service-router configure_shards \
  '(vec {
    principal "shard-1-principal-id";
    principal "shard-2-principal-id";
    principal "shard-3-principal-id";
  })'

Errors:

"Unauthorized" - Not a controller
"Invalid principal: ..." - Invalid canister ID
"Shard list cannot be empty" - Empty vector provided

Warning: This operation is disruptive. Changing shard count invalidates routing and requires data migration.

add_shard

Add new shard to configuration (for capacity expansion).

Signature:

candid

add_shard : (principal) -> (variant { Ok : null; Err : text });

Access: Controller only

Parameters:

shard: principal - New shard canister principal

Returns: Result<(), String>

Behavior:

Validates caller is controller
Validates shard principal
Checks shard not already registered
Appends to shard list
Clears routing cache (routing formula changes)

Example:

bash

# Deploy new shard
dfx deploy user-service-shard-4 --network mainnet

# Register with router
dfx canister call user-service-router add_shard \
  '(principal "new-shard-principal-id")'

Errors:

"Unauthorized" - Not a controller
"Shard already registered" - Duplicate principal
"Invalid principal" - Invalid canister ID

Warning: Adding shards changes routing formula. Requires data rebalancing for optimal distribution.

remove_shard

Remove shard from configuration (for decommissioning).

Signature:

candid

remove_shard : (principal) -> (variant { Ok : null; Err : text });

Access: Controller only

Parameters:

shard: principal - Shard canister principal to remove

Returns: Result<(), String>

Behavior:

Validates caller is controller
Validates shard exists in configuration
Validates at least 2 shards will remain (minimum requirement)
Removes from shard list
Clears routing cache

Example:

bash

dfx canister call user-service-router remove_shard \
  '(principal "shard-to-remove-id")'

Errors:

"Unauthorized" - Not a controller
"Shard not found" - Principal not in configuration
"Cannot remove last shard" - Would leave empty configuration
"Must have at least 2 shards" - Minimum configuration requirement

Warning: Removing shards causes data loss unless data is migrated first.

list_shards

List all configured shard principals.

Signature:

candid

list_shards : () -> (vec principal) query;

Access: Public (read-only)

Returns: vec principal - List of shard canister IDs

Example:

javascript

const shards = await userServiceRouter.list_shards();

console.log(`Configured shards: ${shards.length}`);
shards.forEach((shard, i) => {
  console.log(`Shard ${i}: ${shard.toText()}`);
});

Example (dfx):

bash

dfx canister call user-service-router list_shards

Output:

(vec {
  principal "aaaaa-aa";
  principal "bbbbb-bb";
  principal "ccccc-cc";
})

Monitoring

get_routing_stats

Get routing statistics and metrics.

Signature:

candid

get_routing_stats : () -> (RoutingStats) query;

Access: Public

Returns: RoutingStats

candid

type RoutingStats = record {
  total_requests : nat64;
  single_shard_requests : nat64;
  broadcast_requests : nat64;
  aggregation_requests : nat64;
  cache_hits : nat64;
  cache_misses : nat64;
  total_latency_ms : nat64;
  average_latency_ms : nat64;
  shard_count : nat64;
};

Example:

javascript

const stats = await userServiceRouter.get_routing_stats();

console.log(`Total requests: ${stats.total_requests}`);
console.log(`Cache hit rate: ${
  (stats.cache_hits / (stats.cache_hits + stats.cache_misses) * 100).toFixed(2)
}%`);
console.log(`Avg latency: ${stats.average_latency_ms}ms`);

Metrics Tracked:

total_requests: All requests routed
single_shard_requests: Routed to single shard (email/individual-based)
broadcast_requests: Queries sent to all shards (principal lookups)
aggregation_requests: Results combined from all shards (stats, listing)
cache_hits: Address routing resolved from cache
cache_misses: Required broadcast lookup for address routing
total_latency_ms: Cumulative routing overhead
average_latency_ms: Average overhead per request
shard_count: Current number of configured shards

get_shard_health

Get health status for all shards.

Signature:

candid

get_shard_health : () -> (vec ShardHealth) query;

Access: Public

Returns: vec ShardHealth

candid

type ShardHealth = record {
  shard_id : principal;
  shard_index : nat64;
  status : ShardStatus;
  last_check : nat64;
  response_time_ms : opt nat64;
  error_message : opt text;
  user_count : opt nat64;
  address_count : opt nat64;
};

type ShardStatus = variant {
  Healthy;
  Degraded;
  Unreachable;
  Unknown;
};

Example:

javascript

const health = await userServiceRouter.get_shard_health();

health.forEach(shard => {
  console.log(`Shard ${shard.shard_index}: ${
    'Healthy' in shard.status ? 'OK' : 'ERROR'
  }`);
  if (shard.user_count) {
    console.log(`  Users: ${shard.user_count[0]}`);
  }
  if (shard.response_time_ms) {
    console.log(`  Response: ${shard.response_time_ms[0]}ms`);
  }
});

Health Check Behavior:

Calls get_stats_shard() on each shard
Measures response time
Categorizes status:
- Healthy: Response < 500ms, no errors
- Degraded: Response 500-2000ms, or intermittent errors
- Unreachable: No response or consistent errors
- Unknown: Not yet checked

Refresh Interval: Health checks run every 60 seconds in background

get_request_metrics

Get detailed request metrics by operation type.

Signature:

candid

get_request_metrics : () -> (RequestMetrics) query;

Access: Public

Returns: RequestMetrics

candid

type RequestMetrics = record {
  submit_individual_count : nat64;
  verify_code_count : nat64;
  add_address_count : nat64;
  update_address_count : nat64;
  delete_address_count : nat64;
  get_user_by_principal_count : nat64;
  get_current_user_count : nat64;
  get_stats_count : nat64;
  list_individuals_count : nat64;

  total_errors : nat64;
  routing_errors : nat64;
  shard_errors : nat64;

  uptime_seconds : nat64;
  started_at : nat64;
};

Example:

javascript

const metrics = await userServiceRouter.get_request_metrics();

console.log(`submit_individual: ${metrics.submit_individual_count} calls`);
console.log(`Total errors: ${metrics.total_errors}`);
console.log(`Uptime: ${metrics.uptime_seconds} seconds`);

Use Cases:

Monitor operation frequency
Identify hot paths
Debug error patterns
Capacity planning

Data Types

The router uses the same data types as user-service. See user-service.md for complete reference.

Router-Specific Types

RoutingStats

candid

type RoutingStats = record {
  total_requests : nat64;
  single_shard_requests : nat64;
  broadcast_requests : nat64;
  aggregation_requests : nat64;
  cache_hits : nat64;
  cache_misses : nat64;
  total_latency_ms : nat64;
  average_latency_ms : nat64;
  shard_count : nat64;
};

ShardHealth

candid

type ShardHealth = record {
  shard_id : principal;
  shard_index : nat64;
  status : ShardStatus;
  last_check : nat64;
  response_time_ms : opt nat64;
  error_message : opt text;
  user_count : opt nat64;
  address_count : opt nat64;
};

type ShardStatus = variant {
  Healthy;
  Degraded;
  Unreachable;
  Unknown;
};

RequestMetrics

candid

type RequestMetrics = record {
  submit_individual_count : nat64;
  verify_code_count : nat64;
  add_address_count : nat64;
  update_address_count : nat64;
  delete_address_count : nat64;
  get_user_by_principal_count : nat64;
  get_current_user_count : nat64;
  get_stats_count : nat64;
  list_individuals_count : nat64;
  total_errors : nat64;
  routing_errors : nat64;
  shard_errors : nat64;
  uptime_seconds : nat64;
  started_at : nat64;
};

Error Handling

Router Errors

The router adds routing-specific errors on top of user-service errors:

Routing Errors

Error Message	Cause	Resolution
`"No shards configured"`	Empty shard list	Call `configure_shards` first
`"Shard not found for routing key"`	Invalid routing calculation	Check shard configuration
`"All shards unreachable"`	All broadcast queries failed	Check shard health
`"Routing cache corrupted"`	Internal cache inconsistency	Restart router canister

Shard Configuration Errors

Error Message	Cause	Resolution
`"Unauthorized"`	Caller not controller	Use controller principal
`"Invalid principal"`	Malformed principal	Verify canister ID format
`"Shard already registered"`	Duplicate add_shard	Check list_shards output
`"Shard not found"`	Unknown principal in remove_shard	Verify shard exists
`"Shard list cannot be empty"`	Empty configure_shards	Provide at least 1 shard

Shard Communication Errors

Error Message	Cause	Resolution
`"Shard call failed: timeout"`	Shard not responding	Check shard canister status
`"Shard call failed: canister not found"`	Invalid shard principal	Verify canister exists
`"Shard call failed: out of cycles"`	Shard out of cycles	Top up shard canister
`"Partial broadcast failure"`	Some shards unreachable	Check get_shard_health

User-Service Errors

All user-service errors are passed through unchanged. See user-service.md for details.

Error Response Format

Router errors follow the same Result pattern:

candid

variant { Ok : T; Err : text }

Example Error Handling:

javascript

const result = await userServiceRouter.submit_individual(request, address);

if ('Err' in result) {
  if (result.Err.includes("No shards configured")) {
    console.error("Router not configured");
  } else if (result.Err.includes("Email already registered")) {
    console.error("User exists");
  } else {
    console.error("Unknown error:", result.Err);
  }
}

Frontend Integration

Transparent Migration

The router API is 100% backward compatible with standalone user-service. Migrating requires only changing the canister ID:

Before (Standalone):

javascript

import { idlFactory } from './declarations/user-service';

const userService = Actor.createActor(idlFactory, {
  agent,
  canisterId: 'user-service-standalone-id',
});

After (Router + Shards):

javascript

import { idlFactory } from './declarations/user-service'; // Same interface!

const userService = Actor.createActor(idlFactory, {
  agent,
  canisterId: 'user-service-router-id', // Only change
});

All application code remains unchanged.

Environment Configuration

Use environment variables for deployment flexibility:

typescript

// config.ts
export const USER_SERVICE_CANISTER_ID =
  process.env.VITE_USER_SERVICE_CANISTER_ID ||
  process.env.VITE_USER_SERVICE_ROUTER_ID;

// Development: standalone canister
// Staging: standalone canister
// Production: router canister

.env.development:

bash

VITE_USER_SERVICE_CANISTER_ID=local-user-service-id

.env.production:

bash

VITE_USER_SERVICE_ROUTER_ID=router-canister-id

Health Monitoring

Monitor router health in production:

typescript

async function checkRouterHealth() {
  const health = await userServiceRouter.get_shard_health();

  const unhealthy = health.filter(s =>
    !('Healthy' in s.status)
  );

  if (unhealthy.length > 0) {
    console.warn(`${unhealthy.length} shards unhealthy`);
    // Alert monitoring system
  }
}

setInterval(checkRouterHealth, 60000); // Check every minute

Performance Monitoring

Track routing overhead:

typescript

async function monitorPerformance() {
  const stats = await userServiceRouter.get_routing_stats();

  const cacheHitRate =
    stats.cache_hits / (stats.cache_hits + stats.cache_misses);

  console.log(`Cache hit rate: ${(cacheHitRate * 100).toFixed(2)}%`);
  console.log(`Avg latency: ${stats.average_latency_ms}ms`);

  if (stats.average_latency_ms > 500) {
    console.warn("High routing latency detected");
  }
}

Deployment

When to Use Router vs Standalone

Use Standalone when:

Development and testing environments
User count < 500K
Rapid iteration and debugging required
Local development with dfx
Cost optimization for small deployments

Use Router + Shards when:

Production environment with growth expectations
User count > 500K or projected to exceed
High-availability requirements
Geographic distribution needed
Linear scalability required

Initial Deployment (Router + Shards)

Step 1: Deploy Router Canister

bash

dfx deploy user-service-router --network mainnet

Step 2: Deploy Initial Shards (3 recommended)

bash

dfx deploy user-service-shard-1 --network mainnet
dfx deploy user-service-shard-2 --network mainnet
dfx deploy user-service-shard-3 --network mainnet

Step 3: Initialize Shard Signing Keys

bash

for shard in 1 2 3; do
  dfx canister call user-service-shard-$shard initialize_signing_key \
    --network mainnet
done

Step 4: Configure Router

bash

# Get shard principals
SHARD1=$(dfx canister id user-service-shard-1 --network mainnet)
SHARD2=$(dfx canister id user-service-shard-2 --network mainnet)
SHARD3=$(dfx canister id user-service-shard-3 --network mainnet)

# Configure router
dfx canister call user-service-router configure_shards \
  "(vec {
    principal \"$SHARD1\";
    principal \"$SHARD2\";
    principal \"$SHARD3\";
  })" \
  --network mainnet

Step 5: Verify Configuration

bash

dfx canister call user-service-router list_shards --network mainnet
dfx canister call user-service-router get_shard_health --network mainnet

Capacity: 1.5M users (3 × 500K per shard)

Migration from Standalone to Router

Prerequisites:

Standalone canister deployed and operational
Export all user data
Router and shards deployed
Maintenance window scheduled

Step 1: Export Data

bash

dfx canister call user-service list_individuals \
  --network mainnet > individuals.json

Step 2: Deploy Router Infrastructure See "Initial Deployment" above.

Step 3: Data Migration Script

typescript

// migrate-to-shards.ts
import { readFileSync } from 'fs';
import { Actor, HttpAgent } from '@dfinity/agent';

const individuals = JSON.parse(readFileSync('individuals.json', 'utf8'));
const shards = [shard1Actor, shard2Actor, shard3Actor];

async function migrateToShards() {
  for (const individual of individuals) {
    // Calculate target shard
    const shardIndex = hashEmailToShard(individual.email, shards.length);

    // Import to shard
    await shards[shardIndex].import_individual(individual);

    console.log(`Migrated ${individual.id} to shard ${shardIndex}`);
  }
}

function hashEmailToShard(emailHash: string, shardCount: number): number {
  // Implement same hash algorithm as router
  const hash = calculateHash(emailHash);
  return Number(hash % BigInt(shardCount));
}

Step 4: Validation

bash

# Check shard totals
for shard in 1 2 3; do
  echo "Shard $shard:"
  dfx canister call user-service-shard-$shard get_stats_shard --network mainnet
done

# Check router aggregation
dfx canister call user-service-router get_stats --network mainnet

Step 5: Cutover

Update frontend environment config:

bash

VITE_USER_SERVICE_ROUTER_ID=router-canister-id

Deploy updated frontend
Monitor error rates and latency
Gradual traffic shift (use feature flags)
After validation period (7-14 days), decommission standalone canister

Scaling: Adding Shards

When to Add Shards:

Any shard approaching 400K users (80% capacity)
Query latency increasing
Memory usage > 3.2GB (80% of 4GB limit)

Procedure:

bash

# Deploy new shard
dfx deploy user-service-shard-4 --network mainnet

# Initialize signing key
dfx canister call user-service-shard-4 initialize_signing_key \
  --network mainnet

# Add to router
SHARD4=$(dfx canister id user-service-shard-4 --network mainnet)
dfx canister call user-service-router add_shard \
  "(principal \"$SHARD4\")" \
  --network mainnet

# Verify
dfx canister call user-service-router list_shards --network mainnet

Warning: Adding shards changes routing formula (modulo N). Requires data rebalancing for optimal distribution.

Rebalancing (future feature):

Migrate ~25% of users from existing shards to new shard
Maintain even distribution across all shards
Zero downtime with routing fallback

Performance Characteristics

Latency Breakdown

Operation	Standalone	Router (3 shards)	Breakdown
Queries
get_user_by_principal	100ms	200ms	+100ms (broadcast)
get_current_user	100ms	200ms	+100ms (broadcast)
get_stats	100ms	250ms	+150ms (aggregation)
get_addresses	100ms	150ms	+50ms (routing)
Updates
submit_individual	2s	2.5s	+500ms (routing + call)
verify_code	2s	2.5s	+500ms (routing + call)
add_address	2s	2.5s	+500ms (routing + call)
update_address (cache hit)	2s	2.5s	+500ms (routing + call)
update_address (cache miss)	2s	2.6s	+600ms (broadcast + call)

Latency Components:

Routing calculation: 10ms
Cache lookup: 5ms
Inter-canister call overhead: 20-40ms
Broadcast query (parallel): 50-100ms
Result aggregation: 20-50ms

Throughput Scaling

Configuration	QPS	TPS	Users	Memory
Standalone	1,000	100	500K	4GB
Router + 3 shards	3,000	300	1.5M	12GB
Router + 10 shards	10,000	1,000	5M	40GB

Scaling Properties:

Linear query throughput: N shards = N × QPS
Linear update throughput: N shards = N × TPS
Linear storage: N shards = N × capacity

Cache Performance

Address Routing Cache:

Hit rate (typical): 95-98%
Hit latency: +5ms
Miss latency: +100ms (broadcast lookup)
Cache size: 10,000 entries (LRU eviction)
Memory overhead: ~1MB

Cache Warming:

javascript

// Warm cache on startup for common addresses
async function warmCache() {
  const activeUsers = await getActiveUserIds(); // Last 24h activity

  for (const userId of activeUsers) {
    await userServiceRouter.get_addresses_for_individual(userId);
    // Router caches address_id -> shard mappings
  }
}

Capacity Planning

Single Shard Limits (estimated):

Users: 500,000
Addresses: 2,000,000 (4 per user avg)
Verification codes: 100,000
Heap memory: 4GB
Stable memory: 8GB

Shard Count Recommendations:

User Target	Shard Count	Headroom
100K	1 (standalone)	5x
500K	2 shards	2x
1M	3 shards	1.5x
5M	10 shards	1x
10M	20 shards	1x

Headroom: Capacity buffer for growth and load spikes

Architecture

API References

Source Code

Router Repository: /home/coby/git/user-service-router/
User-Service Repository: /home/coby/git/user-service/
Shard Implementation: /home/coby/git/user-service/src/lib.rs
Shard Tests: /home/coby/git/user-service/tests/shard_pattern_tests.rs

Operations

Changelog

Version 1.0 (2025-11-15)

Initial router API documentation
Documented routing strategies (email-based, individual-based, broadcast, aggregation)
Specified all user-service method proxying behavior
Documented router-specific methods (shard config, monitoring)
Performance characteristics and latency analysis
Deployment procedures and migration guide
Capacity planning recommendations

Last Updated: 2025-11-15 API Version: 1.0 (Planned) Status: Design Complete, Implementation Pending Maintainer: API Team

User-Service-Router API Reference ​

Overview ​

Purpose ​

Key Features ​

Deployment Model ​

Architecture ​

Request Flow ​

Routing Architecture ​

Routing Behavior ​

Strategy 1: Email-Based Routing ​

Strategy 2: Individual-Based Routing ​

Strategy 3: Broadcast Queries ​

Strategy 4: Aggregation Queries ​

Routing Overhead ​

Latency Impact ​

Throughput Scaling ​

User-Service API Methods ​

Individual Management ​

submit_individual ​

verify_code ​

get_current_user ​

get_user_by_principal ​

Address Management ​

add_address ​

update_address ​

delete_address ​

set_primary_address ​

get_addresses_for_individual ​

get_primary_address ​

Statistics ​

get_stats ​

Controller-Only Methods ​

list_individuals ​

list_individuals_paginated ​

initialize_signing_key ​

Router-Specific Methods ​

Shard Configuration ​

configure_shards ​

add_shard ​

remove_shard ​

list_shards ​

Monitoring ​

get_routing_stats ​

get_shard_health ​

get_request_metrics ​

Data Types ​

Router-Specific Types ​

RoutingStats ​

ShardHealth ​

RequestMetrics ​

Error Handling ​

Router Errors ​

Routing Errors ​

Shard Configuration Errors ​

Shard Communication Errors ​

User-Service Errors ​

Error Response Format ​

Frontend Integration ​

Transparent Migration ​

Environment Configuration ​

Health Monitoring ​

Performance Monitoring ​

Deployment ​

When to Use Router vs Standalone ​

Initial Deployment (Router + Shards) ​

Migration from Standalone to Router ​

Scaling: Adding Shards ​

Performance Characteristics ​

Latency Breakdown ​

Throughput Scaling ​

Cache Performance ​

Capacity Planning ​

Related Documentation ​

Architecture ​

API References ​

Source Code ​

Operations ​

Changelog ​

Version 1.0 (2025-11-15) ​

User-Service-Router API Reference

Overview

Purpose

Key Features

Deployment Model

Architecture

Request Flow

Routing Architecture

Routing Behavior

Strategy 1: Email-Based Routing

Strategy 2: Individual-Based Routing

Strategy 3: Broadcast Queries

Strategy 4: Aggregation Queries

Routing Overhead

Latency Impact

Throughput Scaling

User-Service API Methods

Individual Management

submit_individual

verify_code

get_current_user

get_user_by_principal

Address Management

add_address

update_address

delete_address

set_primary_address

get_addresses_for_individual

get_primary_address

Statistics

get_stats

Controller-Only Methods

list_individuals

list_individuals_paginated

initialize_signing_key

Router-Specific Methods

Shard Configuration

configure_shards

add_shard

remove_shard

list_shards

Monitoring

get_routing_stats

get_shard_health

get_request_metrics

Data Types

Router-Specific Types

RoutingStats

ShardHealth

RequestMetrics

Error Handling

Router Errors

Routing Errors

Shard Configuration Errors

Shard Communication Errors

User-Service Errors

Error Response Format

Frontend Integration

Transparent Migration

Environment Configuration

Health Monitoring

Performance Monitoring

Deployment

When to Use Router vs Standalone

Initial Deployment (Router + Shards)

Migration from Standalone to Router

Scaling: Adding Shards

Performance Characteristics

Latency Breakdown

Throughput Scaling

Cache Performance

Capacity Planning

Related Documentation

Architecture

API References

Source Code

Operations

Changelog

Version 1.0 (2025-11-15)