Solver.Planning.Domains

An automated planner in the cloud.

PDDL Solver

Solver as a Service

The solver component of planning.domains is an automated planner and validator in the cloud. You can invoke the software either by sending links to the PDDL files or sending raw PDDL content in JSON format directly to retrieve or validate a plan.

Sending PDDL URLs

Primarily, this functionality demonstrates how to use the remote planner, and is used for debugging the output that may be provided when it fails to find a solution. As examples, see how to solve a simple blocksworld problem (a valid plan should be shown), and see what happens when you feed the planner mismatched problems (the planners error log should be shown).

Sending PDDL Directly

A JSON POST request to the solver and/or validator can be used to retrieve or validate a plan, and extra information will be returned as well. For now, this is just the ground action schema corresponding to each action in the plan computed, and the computed solution cost from the validator. More information may be added in the future, depending on the general interest of those using the service. The following API endpoints are available:

POST solver.planning.domains/solve

POST solver.planning.domains/validate

POST solver.planning.domains/solve-and-validate

The solve and solve-and-validate endpoints require either probID or both domain and problem to be set as parameters. The former corresponds to the problem ID as defined by api.planning.domains and the latter can either be the raw PDDL text, or URLs to the domain and problem file. If URLs are used, then is_url additionally must be set to true. Finally, for the validate endpoint, plan must be set and correspond to the IPC standard plan output (i.e., one ground action per line).

The return value will depend on the endpoint used, but the status attribute will always be set to either ok or error. The result attribute will either be an error message (if appropriate), or contain information about the computed plan and/or validation. In particular, the following attributes may appear as part of the result:

Attribute Description
lengthNumber of actions in the plan.
outputThe planner output.
parse_statusIndication of the plan parsing (e.g., ok).
planList of action objects; each containing name and action (a string of the ground action).
typeThe type of parsing (either simple or full) that was achieved. If simple, then the action objects in plan will just be action names.
costThe plan cost.
val_stdoutVAL standard output.
val_stderrVAL standard error.
val_statusThe return status of VAL (either valid or err).
errorIndication of any VAL error.

Online Example

Aside from the integration into the online editor, you can invoke the planner on the range of domains provided by the API by going to this page. The same page is provided as part of the open source project to quickly test your own planner in the cloud as well.

Javascript Example

As an example, if domText and probText respectively contain the domain and problem PDDL text, then the following Javascript code would print the server response that includes the plan to the console (note that the jQuery library is required for the example):

$.ajax( {url: "http://solver.planning.domains/solve-and-validate",
  type: "POST",
  contentType: 'application/json',
  data: JSON.stringify({"domain": domText,
                        "problem": probText})})
    .done(function (res) {
      if (res['status'] === 'ok') {
        window.alert('Plan found!');
      } else {
        window.alert('Planning failed.');
      }
      console.log(res);
    }
);

Python Example

As another example, the following Python code can be considered a fully functional (albeit unreliable) IPC-style planner (i.e., run using ./planner.py domain.pddl problem.pddl plan.ipc):

import urllib2, json, sys

data = {'domain': open(sys.argv[1], 'r').read(),
        'problem': open(sys.argv[2], 'r').read()}

req = urllib2.Request('http://solver.planning.domains/solve')
req.add_header('Content-Type', 'application/json')
resp = json.loads(urllib2.urlopen(req, json.dumps(data)).read())

with open(sys.argv[3], 'w') as f:
    f.write('\n'.join([act['name'] for act in resp['result']['plan']]))

Limitations

Currently the planner is limited to approximately 500Mb of RAM and has a 10 second time limit. Only one request can be served at a time, so it may take longer than 10 seconds for a response to be sent, and there is no guarantee that any request will be handled properly. In the future, clients will be restricted to sending a maximum of one request at a time: only the most recent request from a single source will be considered. These restrictions are due to the limited nature of the hosted solution for solver.planning.domains, and this is the primary purpose of releasing the framework for others to run their own private planner in the cloud.

Open Source Project

The solver.planning.domains service is meant to be for educational purposes only. If you would like to have a more robust and dedicated solution for solving planning problems as a web service, then the best option is to host your own private version of the solver framework.

The core of solver.planning.domains is released as open source software, and you can find all of the details at the public repository. You can replace the planner used by the editor to solve problems, parse and return different or further information, etc. If you have any feedback on improvements or bugs that exist, then please do let us know by filing a bug report.

Ultra-Agile Competition

Backround

The motivation behind a planner in the cloud is to provide a service for the planning.domains editor. As such, we want to have the most responsive planner possible for solving small problems, while a new domain is being created. To find the best planner among all those available, we rely on the planning community to put forward any planner that can oust the current one used online. Hence, the need for an Ultra-Agile Planning Competition.

Every 2-3 months, if there is at least one other planner put forward to challenge the incumbent, a suite of benchmarks will be run on all planners currently vying for ultra-agile supremacy. The fastest planner will become the new incumbent, and this page will be updated with the results.

The current incumbent planner is SIW+-then-BFSf, a configuration put forward by Nir Lipovetzky, Miquel Ramirez, and Christian Muise from the LAPKT planning framework. The first round of the ultra-agile comparison is set to take place in August, 2015. If you would like to have your planner considered, fork the open source project and make sure that you can run your planner on the heroku servers, then contact Christian Muise to have your planner entered.

Criteria and Rules

The solvers do not need to be open source, but they must work without issue using the solver framework (see the open source project for more details on getting your own solver running in the cloud). The only metric used for comparison will be the IPC time score; measured as the best time divided by your time for every problem, summed over all problems. Naturally, this will favour planners with higher coverage, and focus primarily on time-to-valid solution rather than solution quality.

The one non-standard rule is that the timeout will be capped at 10 seconds and the memory at 500Mb. If the problem cannot be parsed, solved, and returned within those limitations, it is not considered solved in an "ultra-agile" manner.

The benchmark set used for testing will not be shared publicly, but will be a subset of all those available in the domain repository. Any problems that arise when running the planner will be debugged with the planner authors. Finally, for the time being, all benchmarks used for evaluation will be STRIPS.

Difference from the Agile IPC Track?

In the 2014 IPC, an agile track was run. While this was a compelling first step towards promoting fast planners, the motivation of the Ultra-Agile Planning Competition deviates slightly. First, it will (hopefully) serve as a far more fine-grained solution to the question of "what is the fastest planner out there?"; at least on the classical benchmark domains that we tend to see. Second, with a cut-off of 10 seconds, the type of solver that will do well must be efficient in areas not typically considered by contest planners: parsing and preprocessing become paramount, and limited memory must be considered.

The Ultra-Agile Competition is not intended to replace the Sequential Agile track at IPC, but rather complement it by focusing on planning solutions that are light-weight and efficient for many problems. Also, it is far more informal, and will be run in an ad-hoc manner whenever someone wishes to dethrone the current best-in-class solver.