Validator Guide

How to stake and validate tasks as a Validator

This guide provides step-by-step instructions for the entire Validator workflow. By the end you will have successfully staked as a Validator, requested task assignment(s), run a validator script, uploaded loss score(s), and claimed your staking rewards.

Be sure you have completed all pre-requisite tasks.

You can only choose one role per task!

1. Stake FML as Validator from train.flock.io

In order begin receiving validation assignments, you must stake FML as to a task at train.flock.io.

Select task and stake FML

1. Navigate to the Validator tab on the Stake page

2. Select task to validate

3. Enter number of FML tokens to stake

4. Stake FML

Once you’ve confirmed and approved the transaction in your web3 wallet, you will see a box on the Validator tab with your stake details.

2. Get API key

Your API key is required for all remaining Validator steps. You can get it from the web app.

1. Select dropdown in upper right corner of web app

2. Select API

3. On the API page, copy your API key

Once you have your API key, you can proceed to the next step.

NOTE: If you have issues generating an API key try removing any ad blocker extensions and/or clearing your cookies.

Video Tutorial

3. Set up your environment

To set up your environment, run the following commands:

Windows

For Windows users, we suggest installing WSL. Follow the guidance: WSL installation

Mac/Linux

You can install Anaconda via HERE

4. Clone the Validator Repo

The validator repo contains everything you need to run our script. To clone it run:

git clone https://github.com/FLock-io/llm-loss-validator.git

To set up all packages within the project directory

conda create -n llm-loss-validator python==3.10
conda activate llm-loss-validator
pip install -r requirements.txt

5. Run Validation Script

Now that you have staked as a Validator and have your API key you can run the validation script by running the following loop in your command line:

Unix/Linux

cd /src
CUDA_VISIBLE_DEVICES=0 \
bash start.sh \
--hf_token your_hf_token \
--flock_api_key your_flock_api_key \
--task_id your_task_id \
--validation_args_file validation_config.json.example \
--auto_clean_cache False

WSL

cd /src
set CUDA_VISIBLE_DEVICES=0
wsl bash start.sh \
--hf_token your_hf_token \
--flock_api_key your_flock_api_key \
--task_id your_task_id \
--validation_args_file validation_config.json.example \
--auto_clean_cache False

PowerShell

cd /src
$env:CUDA_VISIBLE_DEVICES=0
powershell -File .\start.ps1 `
-hf_token <your_hf_token> `
-flock_api_key <your_flock_api_key> `
-task_id <your_task_id> `
-validation_args_file validation_config.json.example `
-other_args @("--auto_clean_cache", "False")

NOTE: If you would like to customise your validation script, visit this repo https://github.com/FLock-io/llm-loss-validator and check out How to run validation script section

Make sure the HF_TOKEN you passed in havs access to the gated base models like gemma, llama2, and llama3 to succesfully validate models fine-tuned based on those.

Links:

https://huggingface.co/google/gemma-7b

https://huggingface.co/meta-llama/Meta-Llama-3-8B

https://huggingface.co/meta-llama/Llama-2-7b-hf

4. Claim rewards

Reward distribution is triggered every 24 hours at midnight UTC. You can claim your rewards via the web app.

Once you’ve connected your web3 wallet to train.flock.io, you can claim rewards with the following steps:

1. Navigate to the Validator tab on the Stake page

2. Verify you have FML available to claim for the task you are staking

3. Click Claim FML

Once you’ve approved the transaction in your web3 wallet you will see a dialogue box confirming your claim.

Congratulations! You’ve successfully completed the entire validator workflow.