Huygens Benchmarking Suite

Home

Documentation

Registration

CEC2006 Huygens Probe Competition

28 June 2006

The competition is now open! See the CEC2006 link at the new site for details.

21 June 2006

The new Huygens site has now gone live for training and benchmarking!

The new client may now be incorporated in your code and tested. This client will be used for the competition.

16 June 2006

Due to unforseen technical and manpower issues the competition opening on the new server has been delayed for a few days. These should be resolved shortly. Thank you for your patience.

31 May 2006

The Huygens Probe competition will open soon. To improve performance and reporting it will use a client built on the new WebServices-based client, huygensWSClient. More information on this client can be found on the Home page, and the client can be downloaded from the bottom of the Downloads page.

If you have been using the old client, please give the new one a try. As always, let me know of any difficulties.

Introduction

In this competition you will be given access to a series of 20 "moons"- fractal landscapes (generated by sequences of meteor impacts) that are wrapped in both x and y dimensions. Example landscapes can be viewed here. For each moon you will be allowed 1000 probes (evalutations) to find the lowest point on the surface that you can. You may use any technique at your disposal.

The scientific aim is to allow comparison of the many evolutionary (and other more traditional) approaches to real-valued (non-differentiable) optimisation, given a fixed resource in the number of evaluations, and to better understand the cost/benefit trade-offs of population based methods.

How does it Work?

The competition consists of an (optional) training phase, from December 2005 through to May 2006, followed by the competition itself which takes place in June 2006.

During the training phase, prospective contestants are given access to facilities for testing, training and/or fine tuning their algorithm(s). This includes:

Unlimited access to a large set of training moons (more than 2 billion distinct landscapes). These moons are generated in an identical manner to the series that will be used in the competition, but seeded with a different pseudo-random sequence.
Limited acccess to a specified sequence of benchmarking moons from the same sequence as those used for the competition. Attempts at these moons will be scored and the results displayed in the on-line league tables. This enables you to benchmark your algorithm against others that have been submitted.

More information on accessing the moons can be found under Documentation. The Quick Start guide is the easiest way to get started. Additional information on training can be found under Training.

For the competition, access will be given to a new set of 20 unseen moons from the same sequence used for benchmarking. Each individual or group will be allowed to enter up to three algorithms.

Unlike the benchmarking where an absolute score is generated, the competition will use a ranking system where the entries are compared directly against each other for performance on each moon. All entries will be ranked for their performance on each of the 20 moons, and the algorithm with the highest overall average ranking will be the winner.

Important Dates

December 2005: Competition rules published. Access to training and benchmarking landscapes provided.
January 31, 2006: Deadline for CEC 2006 paper submission. Benchmarking scores may be included in papers.
June 21st: Competition client available for testing.
June 23rd: Competition opens. Algorithms may be run at any time.
June 30th: Competition closes.
July 18-22: WCCI 2006. Winner announced at WCCI banquet.