Enterprise RAG Challenge

This is a friendly competition to build a RAG system that can read annual reports and answer questions.

Update: the competition part is over (winner announcement), but we are still running experiments! This experiment tracking server is guaranteed to run at least until April.

• Overview page of challenge
• Registration
• Submission format description
• Optional but recommended:
  • Download sample questions and data.
  • Study source code of the question generator.
  • Generate your own test submission JSON and submit it below for a test.
• Join us on February 27th at 10:00 AM UTC+1 on Discord for challenge data. It will be uploaded here.
• Upload your competition submission and fill in the questionnaire. Anonymous participation is allowed, but you still must fill in the form.
• Multiple submissions from a team are allowed. Only the first submission counts for the prizes, but all submissions with questionnaire go to the leaderboard.

1. Adhere to the Submission Guidelines
2. Check Your Submission
   • Validate your submission format for any issues that could impact scoring.
3. Submit Answers
   • Use the manual submission form below, or
   • Submit via the API.
   • Your submission will be timestamped using TSP. Store the full verification data of the API response or via button after UI submission.
   • Important!Fill in a short form on Tally describing your submission. You will need to paste tsp_signature there.
4. Verify Submission with TSP
   • Use the provided code snippet to verify your submission after uploading.
   • Make sure to store the tcp_verification_data from the UI or API response.
5. Deadlines
   • Prize deadline - March 01, 00:00 UTC+1. Only submissions sent in February can be nominated for the prizes.
   • Leaderboard deadline - March 07, 10:00 UTC+1. Only submissions sent before we release the answers are accepted.
6. Upload multiple submissions
   • Uploading multiple submissions with different architectures and approaches is encouraged! All submissions with a filled form will go to the primary leaderboard.
   • By default, the first submission from the team until the prize deadline (March 01, 00:00 UTC+1) is nominated for the prize. But until the deadline you can replace your nomination with a better but slower solution. Use "Do you want to replace your previous prize submission?" section in the form to do that.