> ## Content Index > Fetch the complete content index at: https://expected-parrot.ghost.io/llms.txt > Use this file to discover other available public pages before exploring further. # I want to see how OpenAI's o3 compares to... - URL: https://expected-parrot.ghost.io/i-want-to-see-how-openais-o3-compares/ - Published: 2025-04-22T21:37:01.000Z - Updated: 2026-10-17T15:55:32.000Z - Description: EDSL is an open-source tool that lets you readily compare performance for many language models at once. - Author: John Horton - Tags: Newsletter, #Migrated-1792252444613, #Import 2026-10-17 15:54 Several key features of [EDSL](https://docs.expectedparrot.com/en/latest?ref=expected-parrot.ghost.io) make it a convenient tool for comparing the performance of language models: - **Unified access**: Connect to multiple language model providers with a single API key and universal methods—no software engineering required - **Structured data collection**: Automatically formatted datasets of responses eliminate manual data cleaning work - **Analysis tools**: Built-in methods for analyzing, reproducing and exporting results ## How it works EDSL is an [open-source Python package](https://github.com/expectedparrot/edsl?ref=expected-parrot.ghost.io) designed to simplify conducting experiments with AI agents and language models. A typical workflow consists of the following steps: - Create [questions](https://docs.expectedparrot.com/en/latest/questions.html?ref=expected-parrot.ghost.io) (free text, multiple choice, numerical, matrix, etc.) - Combine questions in [surveys](https://docs.expectedparrot.com/en/latest/surveys.html?ref=expected-parrot.ghost.io) with custom logic (skip patterns, stop rules, etc.) - (*Optional*) Create personas for AI [agents](https://docs.expectedparrot.con/en/latest/agents.html?ref=expected-parrot.ghost.io) to answer the survey - Select [language models](https://docs.expectedparrot.com/en/latest/language%5Fmodels.html?ref=expected-parrot.ghost.io) to generate the responses - Analyze [results](https://docs.expectedparrot.com/en/latest/results.html?ref=expected-parrot.ghost.io) in specified datasets ## Simplified access to models EDSL is designed for researchers who want to work with language models without spending lots of time on software development. It eliminates the need for one-off coding required to access individual service providers by letting you connect to them all in the same straightforward way. ![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/2dcb2677-cb7f-4349-9013-573df69ea9c2_960x464.png) In EDSL, you can access models individually or collectively by simply selecting them and specifying desired parameters. For example, here we create a list of OpenAI models in order to use them all simultaneously with a survey when we run it. We can inspect the default parameters for the models that will be used: ![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/a320ff30-820b-4837-8f45-fe682037d325_1958x782.png) You can also create multiple instances of the same model with different parameters. Here we create a new model list where we adjust the temperature of a single model: ![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/c38ec8c1-1d49-4bd8-9941-988abf580eab_1960x652.png) ## Results as formatted datasets Running a survey with multiple models at once allows us to readily compare responses in the dataset of results that is generated. Here we run a simple survey (consisting of a linear scale question and a numerical question—see [examples of all question types](https://docs.expectedparrot.com/en/latest/questions.html?ref=expected-parrot.ghost.io)) and inspect the models’ responses: ![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/575e8793-7845-4bff-b848-169824d8fbb7_1960x1396.png) [![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/8b18a505-2603-4452-932b-fee39acb9286_1958x640.png)](https://www.expectedparrot.com/content/RobinHorton/o3-optimist-temps?ref=expected-parrot.ghost.io) These results can be viewed at Coop as well: [![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/bfeb9528-40f3-45ac-97bf-997e8b9e330a_2804x1648.png)](https://www.expectedparrot.com/content/RobinHorton/openai-optimism?ref=expected-parrot.ghost.io) [![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/0f12ff29-686b-4390-bce8-1b98d2ff7b0c_2808x1622.png)](https://www.expectedparrot.com/content/RobinHorton/o3-optimist-temps?ref=expected-parrot.ghost.io) Other columns of the [results](https://docs.expectedparrot.com/en/latest/results.html?ref=expected-parrot.ghost.io) include: - User and system prompts - Question details - Agent traits and instructions - Model temperature and other settings - Raw and formatted responses - Log probs - Input and output tokens - Costs ## Works with many models EDSL works with many other popular [language models](https://docs.expectedparrot.com/en/latest/language%5Fmodels.html?ref=expected-parrot.ghost.io) as well. You can check available models, current token prices and daily performance on test questions [here](https://www.expectedparrot.com/models?ref=expected-parrot.ghost.io): [![](https://storage.ghost.io/c/bc/8c/bc8c40e2-e3d8-4888-8bb9-3276553ff8a4/content/images/2026/10/02b5f750-d06a-41d1-b618-e01267be6192_2900x1672.png)](https://www.expectedparrot.com/models?ref=expected-parrot.ghost.io) *Please get in touch to request other service providers that you want to use!* ## Getting started Our [documentation page](https://docs.expectedparrot.com/en/latest?ref=expected-parrot.ghost.io) includes tutorials and demo notebooks for getting started using EDSL with language models, including examples of methods for [evaluating model responses](https://docs.expectedparrot.com/en/latest/notebooks/models%5Fscoring%5Fmodels.html?ref=expected-parrot.ghost.io). *If you have a use case you don’t see, please see us a message and we’ll create a notebook for you!* [Subscribe](#/portal/signup)