Hackathon Lessons — and What They Mean for Hiring AI Talent in 2026

A behind-the-scenes look at the KRAFTON AI R&D Hackathon · Spring 2026 · ~300 participants · 5 problems · 2 weekends

We just wrapped up the first KRAFTON AI R&D Hackathon — a two-week, two-round contest with about 300 participants from across Korea. The crowd was wonderfully mixed: undergraduates, PhD students, AI researchers, non-engineers, and even tech leads from some of Korea’s most prominent companies, all chasing the same problems.

This post is the long version of “what did we make people do, and why?” The first half is the problem set itself — rules, time limits, and links to the original statements — for anyone who wants to try the problems cold. The second half, after a clearly marked spoiler line, is the behind-the-scenes story: where each problem came from, what the intended solutions were, the surprises in the submissions, and what designing this whole thing taught me about hiring AI talent in 2026.

Round 1 — Online · 2 Days · 4 Hours Each

Round 1 ran on a single weekend, fully online. Two problems, one per day, 4 hours each. Anything goes. Any tools, any AI, any libraries.

Day 1 · Problem 1 — MultiplierBoard

Inspired by the popular AdderBoard challenge from @DimitrisPapail, which asked the same question for addition. Multiplication is meaningfully harder (carries propagate non-locally and partial products interact) so the design space is much richer.

Full problem statement: kangwooklee.com/blogs/krafton_ai_hackathon_2026/round1_p1.html

Day 2 · Problem 2 — SparseTap

Hidden parameters: a set of offsets d₁ < d₂ < … < dₛ with S ≤ 16 and dₛ ≤ 64. Each new bit equals the XOR of seq[n − dₖ] across those offsets, plus a Bernoulli(0.2) noise bit. Same hidden taps for all 2,000 sequences; different random seeds.

Full problem statement: kangwooklee.com/blogs/krafton_ai_hackathon_2026/round1_p2.html

Round 2 — In Person · Top 30 Finalists

The top 30 from Round 1 were invited to a single in-person weekend in Seoul. The Round 2 problems were structured very differently — more on why in the second half.

Day 1 Morning · Problem 1 — Pen and paper, no devices

1. Self-attention by hand (15 pt). Concrete W_k, W_v, W_q and two 4-dim inputs. Compute keys, values, queries, dot-product attention scores, softmax weights, outputs — with and without causal masking. Plus a True/False on whether next-token prediction requires causal attention.
2. Sampling strategies (10 pt). Greedy / random / top-k / top-p (nucleus) on a tiny example distribution.
3. Learning Parity with Noise (15 pt). Derive Pr[b̃ = ⟨ã, s⟩] = (1 + (1 − 2η)^s) / 2 for the XOR of s LPN samples, prove it from scratch, then plug in η = 0.1, s = 3.

Full problem: kangwooklee.com/blogs/krafton_ai_hackathon_2026/round2_p1.html

Day 1 Afternoon · Problem 2 — BattlePredict

The catch is in the dataset:

• Days 1, 11, and 21 are fully labeled.
• Days 2–10 and 12–20 appear as 18 anonymous day-blocks, 50 matches each, in original order — but with the day label stripped out. You see the block boundaries, you don’t know which block is which day.
• Days 22–50 are the test set, given only as gauntlet orderings (you know who plays whom; you predict the outcomes).

Score: normalized sum of absolute errors over the 10 per-player kill counts. Lower is better.

Full problem statement: kangwooklee.com/blogs/krafton_ai_hackathon_2026/round2_p2.html

Day 2 · Problem 3 — VideoAgent (5 hours)

Full problem statement: kangwooklee.com/blogs/krafton_ai_hackathon_2026/round2_p3.html

Why have an AI hackathon at all these days?

Honestly, my biggest worry going in was a worry about hiring itself. In 2026, almost every strong candidate is going to hand you something that looks great. Agents write the code, agents write the report, agents debug the agents. Agents write a fancy CV and papers for you. So what do you actually evaluate?

I came in with two beliefs that ended up shaping every problem.

Round 1 problems

First: if a candidate doesn’t use AI to attack hard problems, they’re already out. I don’t want to hire someone who’s allergic to the most powerful tool of our generation. So we kick off the contest with some problems that are extremely hard yet comfortably solvable by current AI agents.

(Note: We reimbursed every participant up to $200 to spend on anything: cloud, GPUs, API credits, model subscriptions!)

Round 2 problems

Second: but for the problems that matter, the human still has to know what they’re doing. Those are the cases where the AI output looks plausible and is subtly wrong, and you need someone who can smell the rot. So the hardest problems were deliberately designed not to fall to current SOTA agents.

That tension (lean on AI hard, but understand it deeply) is the entire spine of the problem set.

Round 1, Problem 1 — MultiplierBoard

This one was a love letter to AdderBoard. AdderBoard asks: what is the smallest transformer that can add? We just turned the crank one notch and asked the same question for multiplication. We split it into the same two flavors:

• Hand-coded weights with a correctness proof. This is the part where the human has to understand attention. Which head attends to which bit, what each MLP computes, how partial products get summed. A wall of weights with no proof gets zero credit.
• Trained weights under a frozen training protocol. Here all you control is the architecture; we run the optimizer, schedule, and decoding.

The boundary between “clever weight construction” and “you basically wired a multiplication circuit and called it a transformer” is blurry. We graded generously. If you could explain the construction layer by layer and the math checked out, we credited it.

The fun surprise: for the training track, a huge fraction of submissions used autoresearch-style agent frameworks to search the architecture space. People let an agent propose architectures, train them under the fixed protocol, and iterate on the loss curve. Exactly the kind of “use AI to attack the search problem” behavior we wanted to reward.

Round 1, Problem 2 — SparseTap (the trap)

The cover story is “find the hidden XOR taps and predict the next bits.” If you squint, it looks like a signal-processing or LFSR-cracking problem. Lots of people went down that road and tried to brute-force the offsets, fit a gradient-descent model on {0,1} outputs, and so on.

The intended path: this is just LPN

The key observation is that this problem is literally Learning Parity with Noise (LPN) in disguise:

• The “secret” is the binary indicator vector of which offsets are taps: s ∈ {0,1}^W with at most S ones.
• Each “sample” is (a, b) where a is a window of W previous bits and b is the next bit — i.e., b = ⟨a, s⟩ ⊕ e with e ∼ Bernoulli(0.2).
• Bits are flipped independently with η = 0.2. That’s LPN.

Once you see this, the textbook attack is the Blum–Kalai–Wasserman (BKW) algorithm: XOR-reduce the dataset by combining samples whose a vectors collide on chosen bit blocks, trading sample count for noise reduction. The bias shrinks as (1 − 2η)^s — the exact identity we put on the Round 2 written exam, as a bridge.

The unintended-but-also-correct path: RANSAC

Several people used a RANSAC-style approach I hadn’t planned for. The idea: randomly sample a small subset of (a, b) constraints, assume they’re all noise-free, solve the resulting linear system over GF(2), and check how many of the remaining constraints are satisfied. If a lot, you’ve found s.

I was initially surprised this works, but the math actually checks out at the specific scales of this problem:

P(single sample is noise-free)         = 1 − 0.2 = 0.8
P(random S-subset is fully noise-free) = 0.8^S    ≈ 0.8^16 ≈ 0.028
                                                            ≈ 1 in 35
Total constraints in dataset            = 2000 × 192 ≈ 384,000

⇒ Sampling random S-subsets and verifying is comfortably tractable.
   RANSAC + a GF(2) solver + verification recovers s in seconds.

RANSAC works on this problem precisely because S, η, and the dataset size happen to be in a friendly regime. Squeeze any one of them and the trick collapses.

I had not designed for this and I love that it worked. Two very different solutions, one I planned and one the participants taught me.

After Round 1

After grading Round 1, we picked the top 30 to advance. Diverse group: students, published AI researchers, non-engineers, several tech leads from well-known Korean companies.

Almost every Round 1 submission looked great. Not just “passed the bar”… actually impressive. Slick code, decent reports, reasonable accuracy. The agents are very good now. Differentiating people purely from their submissions was hard.

If the deliverables all look comparable, what are you actually selecting on? In Round 2 I wanted to give bonus points to people who actually understand what they are doing: not as a gate, but as a tiebreaker that surfaces the deeper kind of skill we would most miss when AI inevitably gets something wrong.

Every Round 2 problem also went through the red-teaming pass: except the pen-and-paper exam, which is closed-book by design and has no agents at the table anyway. For the device-allowed problems, I burned a lot of compute iterating between problem creation and stress-testing with Claude Code, ChatGPT Pro, and autoresearch-style agents, confirming that none of them were solvable end-to-end by any single SOTA call. Each one required real human design choices on top of whatever the agent could do. Write a problem, throw it at every SOTA agent I had access to, watch where it broke, harden the problem, repeat. That iteration loop was, embarrassingly, the most fun I had in the whole two weeks.

Round 2, Day 1 Morning — the pen-and-paper exam

This is the bonus-points problem. No devices, no AI, two hours. The entire exam is structured as a pair of bridge questions for the two Round 1 problems.

Q1 (self-attention by hand) and Q2 (sampling) are the bridge for MultiplierBoard. If you genuinely hand-constructed a transformer that multiplies 6-bit numbers, hand-computing softmax attention on a 2×2 toy and applying nucleus sampling to a 3-token distribution should be mechanical.

Q3 (the LPN noise-amplification identity) is the bridge for SparseTap. It asks how the noise parameter compounds when you XOR samples together — which is the core operation behind BKW. Almost every Round 1 SparseTap submission mentioned and used BKW in their report, so Q3 was checking whether participants actually understood the math behind the XOR-reduction step they were relying on.

So the morning exam is, in effect, a sanity check: did you really do the Round 1 problems yourself, or did an agent do them and you don’t quite know what it built?

A meaningful number of the top 30 (people who had aced Round 1) could not solve the basic attention/sampling questions at all. These were candidates whose Round 1 submissions had been brilliant.

I don’t quite know what to do with that finding yet. On one hand, the role of “knows attention numerically by hand” has objectively shrunk as tools have improved. Maybe it’s fine. On the other hand, the role of “can tell when the AI’s answer is subtly wrong” has objectively grown. Historically those two skills have been correlated. Whether that correlation still holds in 2026 is, I think, the central open question of AI hiring right now.

Round 2, Day 1 Afternoon — BattlePredict

From here on out, anything goes. Bring whatever AI you want. (Reminder: we paid for it.)

This problem comes straight from work my team is actually doing at KRAFTON. PUBG is a battle royale, which means skill rating at scale, which means dealing with the fact that player skill is non-stationary: people warm up, get bored, take breaks, learn, decay. Any honest MMR system has to handle that. So I built the smallest possible toy version of that real problem and turned it into a contest.

The setup

Ten users, each with a linearly time-varying skill s_i(d) = w_i · d + b_i over 50 days. Fifty days, 50 matches per day, 5 players per match, BTL outcomes. Activity rates also vary in time, so heavy users on day 3 may be ghosts by day 18.

The twist: Days 1, 11, and 21 are fully labeled. Days 2–10 and 12–20 are present but with the day labels stripped out, organized into 18 anonymous day-blocks. Predict each player’s total kills on days 22–50.

The intended pipeline

Each step requires you to actually understand the problem. Step 2 is the gating insight: if you skipped it and ran BTL on the full data assuming stationarity — as most blind agent runs did — you would be confidently wrong from there on out. Step 4 uses the fact that you have linear-trend anchors at days 1, 11, 21, so you can compute, for each anonymous block, a cost matrix of “how well does this block match the interpolated skills for day d?” and solve as a linear assignment problem with the Hungarian method. Once you have the recovered labels, the per-user skill curves come out beautifully linear, and extrapolation to days 22–50 is straightforward.

Internal baselines

We computed several reference baselines after the contest. Lower scores are better:

What happens if you just hand it to an agent?

Almost universally, the agent grabs the data with day labels (days 1, 11, 21), fits a BTL model on those ~600 duels, and reports the answer. It doesn’t look at the anonymous day-blocks at all. It doesn’t notice that even the three labeled days disagree with each other on per-user skill. It doesn’t realize the skills are non-stationary, let alone try to recover the hidden day labels. The output is confident, plausible-looking, and exactly the naive baseline.

The successful human competitors did the opposite. They started with data exploration — and they used AI heavily for the exploration itself, asking the agent to plot per-day skill estimates from the labeled days, visualize the duel and activity distributions across the anonymous blocks, and generate diagnostics on the fly. Once the picture made the non-stationarity obvious, the rest of the pipeline (per-block BTL → Hungarian → re-fit → extrapolate) became a series of small, well-scoped instructions the agent could execute correctly. They weren’t doing the math by hand… they were using the agent as a microscope, then telling it where to point.

This was my favorite kind of problem: the agent can do every individual step if asked, but it won’t put them in the right order on its own. The bottleneck was never the agent’s capability. It was knowing what to ask for next.

Round 2, Day 2 — VideoAgent

The grand finale. Five hours. Build an automated video QA agent. We hand you a folder of 20 videos and 20 prompts at T − 15 minutes; you have to submit your 20 answers by T.

Design choices that mattered

Why video QA specifically? Because, today, video QA is exactly the kind of task where naively throwing a SOTA multimodal model at the input does NOT work AT ALL for hard instances. There were really two viable paths. You could hand-design the agent harness yourself (e.g., frame sampling, audio extraction, multi-pass reasoning, verification) and pick the model by hand. Or you could go meta-harness (arxiv.org/abs/2603.28052) and let an outer optimizer use AI to tune the harness for you. The catch with the second path: you had to build your own evaluation set first, because participants had no access to the test set or any training data (and without an eval set), the meta-optimizer has no hill to climb. Either way, real human work was required.

Result

Despite the difficulty and openness of the problem, the top entry scored close to 50% on the completely hidden test set from a 5-hour cold start with no access to the test set or any training set. There were many other interesting solutions in the middle of the pack: different decomposition strategies, different fallback heuristics, different ways of cross-checking pass-by-pass. The top entry was a clear standout.

What I’m taking away

A few thoughts after grading several hundred submissions over two weekends:

The right hire in 2026 is someone who reaches for AI by default and overrides it on purpose. Both halves matter. The contest is, in some sense, an attempt to measure both halves at once.

The top three winners walked away with a healthy cash prize, and I walked away with a much more interesting and unsettled set of opinions about hiring than I came in with. That feels right for a first edition. Beyond the winners, a lot of participants told us this was the most exciting AI hackathon they had ever joined, and that they found it deeply educational — honestly, that was the most rewarding part.

Thanks to every participant who showed up, stayed up, and shipped. We’ll do this again next year (maybe sooner?) and I promise the problems will be even weirder.

Hope you enjoyed :-)

Acknowledgements

Thank you to the ~300 participants who joined Round 1, the 30 finalists who came to Seoul, and the entire KRAFTON AI R&D team that ran logistics, grading, and the on-site weekend.

Special thanks to Prof. Dimitris Papailiopoulos for the original AdderBoard challenge that inspired Round 1, Problem 1.

KRAFTON AI R&D Hackathon · Spring 2026 · Retrospective