Proof of Work · Grading

How grading works

A Result is only worth reading if the numbers on it are hard to fake. This page says where each one comes from, and where the record stops.

One run, start to finish

01Browser
Run tests
The editor sends the code to the server. Nothing is graded in the browser.
02Server
Syntax check
Code that does not parse comes back with the line number. No tests run.
03Isolated sandbox
Sandbox run
The code and the test suite run together in a remote sandbox, never on the app server. A run that takes longer than 15 seconds is stopped.
04Server
Verdict
The server reads one result line from the run: each test name, pass or fail. Every required test must pass.
05Server
Signed record
The server adds this run to a signed record kept in a cookie. The browser carries it but cannot edit it.

If the sandbox itself is down, the run is not graded, not counted and not saved. An outage never shows up on a Result as a failed attempt.

The tests

A candidate sees the name of every test before writing a line. The test code stays on the server. Required tests decide the verdict. Bonus tests are reported and never change it.

Each test loads a fresh copy of the candidate's code. That code cannot print a result of its own or read the test source. The runner also keeps its own references to the built-in functions it relies on, and a test fails if the code has replaced one of them.

The clock

Tasks such as debounce and retry depend on time. Inside the sandbox, setTimeout, clearTimeout and Date.now run on a clock the tests control. Time moves only when a test moves it, so a 100 ms wait is checked exactly and instantly, and the same code gets the same verdict every run.

setInterval is not available in a Challenge.

What the signed record holds

The record is updated only after the sandbox has graded a run, and it is signed with a key the browser never sees. A candidate can clear it and start over. They cannot write a better history into it.

Graded runs: How many times the sandbox graded code for this Challenge.
Distinct versions: How many different versions those runs covered. Running the same code twice is two runs and one version.
First run to save: The time from the first graded run to the save. It is not time spent typing.
First run verdict: Whether the first graded run already passed every required test.
Largest addition: The biggest increase in code size between two graded runs, in bytes. The starter code counts as the step before run one.
Each run: For up to 60 runs: seconds since the first run, bytes added or removed since the previous run, and pass or fail. The Result draws these as bars.

Saving a Result

Save grades the code once more, so the verdict on the Result belongs to the exact code on the Result. If that code is the version the last run already graded, the save does not add a run. A program that never reached the tests cannot be saved.

Where the record stops

Keystrokes, pastes, focus changes or anything else between runs.
The code of earlier versions. The record keeps a size and a fingerprint of each graded version, and only the saved code is stored.
Camera, microphone or screen.
Who was at the keyboard.

The counts describe how the code reached the grader. They do not say who wrote it. See what a Result shows for what an Employer can conclude from one.

Try a Challenge

Open a real sample ResultWritten by Portfolio Pro to show the format, not by a candidate.

Portfolio Pro

Loading page…

One run, start to finish

01Browser

Run tests

The editor sends the code to the server. Nothing is graded in the browser.

02Server

Syntax check

Code that does not parse comes back with the line number. No tests run.

03Isolated sandbox

Sandbox run

The code and the test suite run together in a remote sandbox, never on the app server. A run that takes longer than 15 seconds is stopped.

04Server

Verdict

The server reads one result line from the run: each test name, pass or fail. Every required test must pass.

05Server

Signed record

The server adds this run to a signed record kept in a cookie. The browser carries it but cannot edit it.

If the sandbox itself is down, the run is not graded, not counted and not saved. An outage never shows up on a Result as a failed attempt.

The tests

A candidate sees the name of every test before writing a line. The test code stays on the server. Required tests decide the verdict. Bonus tests are reported and never change it.

The clock

setInterval is not available in a Challenge.

What the signed record holds

The record is updated only after the sandbox has graded a run, and it is signed with a key the browser never sees. A candidate can clear it and start over. They cannot write a better history into it.

Graded runs

How many times the sandbox graded code for this Challenge.

Distinct versions

How many different versions those runs covered. Running the same code twice is two runs and one version.

First run to save

The time from the first graded run to the save. It is not time spent typing.

First run verdict

Whether the first graded run already passed every required test.

Largest addition

The biggest increase in code size between two graded runs, in bytes. The starter code counts as the step before run one.

Each run

For up to 60 runs: seconds since the first run, bytes added or removed since the previous run, and pass or fail. The Result draws these as bars.

Where the record stops

Keystrokes, pastes, focus changes or anything else between runs.

The code of earlier versions. The record keeps a size and a fingerprint of each graded version, and only the saved code is stored.

Camera, microphone or screen.

Who was at the keyboard.

The counts describe how the code reached the grader. They do not say who wrote it. See what a Result shows for what an Employer can conclude from one.