More actions
No edit summary |
update SDE to new place and format |
||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
CCP provides developers a series of static files, known as the Static Data Export(SDE), which contains static data from the Tranquility server. The SDE is currently exported as .yaml and . | {{Update|2025-09-22: CCP has reworked the SDE ([https://developers.eveonline.com/blog/reworking-the-sde-a-fresh-start-for-static-data Reworking the SDE: a fresh start for static data]). This rework is '''not backwards compatible'''.}} | ||
CCP provides developers a series of static files, known as the Static Data Export(SDE), which contains static data from the Tranquility server. The SDE is currently exported as .yaml and .jsonl files, however, different and easier to use formats(mentioned below) are made by the community. The SDE can be found at the EVE developers [https://developers.eveonline.com/static-data Static Data] page. All resources provided by CCP are subject to the [https://developers.eveonline.com/license-agreement developer license agreement]. | |||
== Fuzzwork SDE Conversions == | == Fuzzwork SDE Conversions == | ||
To aide fellow developers and players in consuming this data without having to extract or convert from YAML or | To aide fellow developers and players in consuming this data without having to extract or convert from YAML or JSON Lines format every time Steve Ronuken has hosted conversions in PostgreSQL, SQLite, MySQL, MSSQL, and CSV formats. They can be found at [https://www.fuzzwork.co.uk Fuzzwork] under SDE or more directly at [https://www.fuzzwork.co.uk/dump www.fuzzwork.co.uk/dump]. Individual table data can be found in [[:wikipedia:Comma-separated values|CSV]] and [[:wikipedia:SQL|SQL]] (MySQL) formats here [https://www.fuzzwork.co.uk/dump/latest www.fuzzwork.co.uk/dump/latest]. | ||
=== Opening a file from Fuzzwork SDE Conversions === | === Opening a file from Fuzzwork SDE Conversions === | ||
| Line 7: | Line 10: | ||
== How does the ESI fit into all this? == | == How does the ESI fit into all this? == | ||
The [[EVE Swagger Interface]] intends to have endpoints to account for all of the SDE | The [[EVE Swagger Interface]] intends to have endpoints to account for all of the SDE; currently, it's not quite there yet, and the SDE is needed for many projects. The progress can be tracked on the [https://github.com/esi/esi-issues/issues/1103 SDE parity checklist] at the [https://github.com/esi ESI] GitHub. | ||
== How to implement the original SDE into your application == | == How to implement the original SDE into your application == | ||
=== Step 0: Preface === | === Step 0: Preface === | ||
This is not a guide to Python or programming, if you don't know how to program, there are tons of resources to learn to code, there are a few of my favorites in the External | This is not a guide to Python or programming, if you don't know how to program, there are tons of resources to learn to code, there are a few of my favorites in the [[#External links|External links]] section. | ||
=== Step 1: Opening the file === | === Step 1: Opening the file === | ||
Firstly, find the file containing the data you need, for this example, I'm going to use fsd/blueprints.yaml. This file contains all the information required to build an industry cost predictor. Next, open the file using your desired programming language, I'm going to use python, so to open this I'll run | Firstly, find the file containing the data you need, for this example, I'm going to use fsd/blueprints.yaml. This file contains all the information required to build an industry cost predictor. Next, open the file using your desired programming language, I'm going to use python, so to open this I'll run <code>with open("fsd/blueprints.yaml", "r") as f:</code>. The file is open! To the next step! | ||
=== Step 2: Reading the file === | === Step 2: Reading the file === | ||
==== Step 2.1: Installing PyYAML ==== | ==== Step 2.1: Installing PyYAML ==== | ||
Assuming you're using pip, this should be as simple as | Assuming you're using pip, this should be as simple as <kbd>pip install pyyaml</kbd>. If you having issues have a look at [https://stackabuse.com/reading-and-writing-yaml-to-a-file-in-python/ Reading and Writing YAML to a File in Python] on stackabuse. | ||
==== Step 2.2: Interpreting the file ==== | ==== Step 2.2: Interpreting the file ==== | ||
Now that PyYAML is installed, we need to use it to parse our file, we have already opened it, so it should just be a matter of parsing it with PyYAML, which can be done by | Now that PyYAML is installed, we need to use it to parse our file, we have already opened it, so it should just be a matter of parsing it with PyYAML, which can be done by <code>blueprints = yaml.load(f, Loader=yaml.FullLoader)</code>, remember to have ''import yaml'' at the top of your .py file. Hopefully now if you print out the blueprints variable, your console will be filled with sweet, sweet data and the taste of success. The data will be a dictionary, and the rest from here should be relatively simple, it should just be navigating through the data as if it was a dictionary. | ||
=== Step 3: Postface === | === Step 3: Postface === | ||
| Line 653: | Line 655: | ||
* [https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g Corey Schafer] | * [https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g Corey Schafer] | ||
* EVE Online Discord Dev Chat [https://discord.com/channels/940573867192221696/972841377798946896 3rd-party-dev-and-esi] - I read most messages sent here, if you need help that StackOverflow can't provide, here is your best bet. | * EVE Online Discord Dev Chat [https://discord.com/channels/940573867192221696/972841377798946896 3rd-party-dev-and-esi] - I read most messages sent here, if you need help that StackOverflow can't provide, here is your best bet. | ||
[[Category:Applications]] | [[Category:Applications]] | ||
Latest revision as of 07:38, 28 November 2025
Reason: 2025-09-22: CCP has reworked the SDE (Reworking the SDE: a fresh start for static data). This rework is not backwards compatible.
CCP provides developers a series of static files, known as the Static Data Export(SDE), which contains static data from the Tranquility server. The SDE is currently exported as .yaml and .jsonl files, however, different and easier to use formats(mentioned below) are made by the community. The SDE can be found at the EVE developers Static Data page. All resources provided by CCP are subject to the developer license agreement.
Fuzzwork SDE Conversions
To aide fellow developers and players in consuming this data without having to extract or convert from YAML or JSON Lines format every time Steve Ronuken has hosted conversions in PostgreSQL, SQLite, MySQL, MSSQL, and CSV formats. They can be found at Fuzzwork under SDE or more directly at www.fuzzwork.co.uk/dump. Individual table data can be found in CSV and SQL (MySQL) formats here www.fuzzwork.co.uk/dump/latest.
Opening a file from Fuzzwork SDE Conversions
To extract data from compressed files, use tar for .tar.bz2 files or bunzip2 for .bz2 files in Linux, and something like 7-zip for either format in Windows. Once extracted the files can be opened in the appropriate application that supports the file format.
How does the ESI fit into all this?
The EVE Swagger Interface intends to have endpoints to account for all of the SDE; currently, it's not quite there yet, and the SDE is needed for many projects. The progress can be tracked on the SDE parity checklist at the ESI GitHub.
How to implement the original SDE into your application
Step 0: Preface
This is not a guide to Python or programming, if you don't know how to program, there are tons of resources to learn to code, there are a few of my favorites in the External links section.
Step 1: Opening the file
Firstly, find the file containing the data you need, for this example, I'm going to use fsd/blueprints.yaml. This file contains all the information required to build an industry cost predictor. Next, open the file using your desired programming language, I'm going to use python, so to open this I'll run with open("fsd/blueprints.yaml", "r") as f:. The file is open! To the next step!
Step 2: Reading the file
Step 2.1: Installing PyYAML
Assuming you're using pip, this should be as simple as pip install pyyaml. If you having issues have a look at Reading and Writing YAML to a File in Python on stackabuse.
Step 2.2: Interpreting the file
Now that PyYAML is installed, we need to use it to parse our file, we have already opened it, so it should just be a matter of parsing it with PyYAML, which can be done by blueprints = yaml.load(f, Loader=yaml.FullLoader), remember to have import yaml at the top of your .py file. Hopefully now if you print out the blueprints variable, your console will be filled with sweet, sweet data and the taste of success. The data will be a dictionary, and the rest from here should be relatively simple, it should just be navigating through the data as if it was a dictionary.
Step 3: Postface
I hope this short guide helped you somewhat get into the SDE.
Content of each table (version 2025-07-07)
| Table name | Content |
|---|---|
| agtAgents.csv.bz2 | Contains information regarding agents in NPC stations such as location, corporation, type, level, and locator (boolean) agentID - defined ID for this table |
| agtAgentsInSpace.csv.bz2 | Contains information regarding agents in space agentID - defined ID for this table |
| agtAgentTypes.csv.bz2 | Defines agent types agentTypeID - defined ID for this table |
| agtResearchAgents.csv.bz2 | Defines skills for each Research Agent agentID - the ID of the agent. Corresponds to itemID in invNames |
| certCerts.csv.bz2 | Defines in-game certificates certID - defined ID for this table |
| certMasteries.csv.bz2 | Defines the level of Mastery for flying ships typeID - the ID of the ship. Corresponds to typeID in invTypes |
| certSkills.csv.bz2 | Defines skills and skill levels needed to receive certificates certID - the ID of the certificate. Corresponds to certID in certCerts |
| charFactions.csv.bz2 | Empty file |
| chrAttributes.csv.bz2 | Defines character attributes attributeID - defined ID for this table |
| chrBloodlines.csv.bz2 | Contains information on what race each bloodline belongs to, descriptions. bloodlineID - defined ID for this table |
| chrFactions.csv.bz2 | Faction IDs, name, and description. Also what race each faction belongs to, and some other misc. information. factionID - defined ID for this table |
| chrRaces.csv.bz2 | Description of Races raceID - defined ID for this table |
| crpActivities.csv.bz2 | Description of activities of NPC corporations activityID - defined ID for this table |
| crpNPCCorporationDivisions.csv.bz2 | Description of divisions of NPC corporations corporationID - ID of NPC corporations. Corresponds to itemID in invNames |
| crpNPCCorporationResearchFields.csv.bz2 | Skills available to Research Agents in NPC corporations skillID - the ID of skills. Corresponds to typeID in invTypes |
| crpNPCCorporations.csv.bz2 | Describes NPC corporations corporationID - ID of NPC corporations. Corresponds to itemID in invNames |
| crpNPCCorporationTrades.csv.bz2 | Items sold by NPC corporations corporationID - ID of NPC corporations. Corresponds to itemID in invNames |
| crpNPCDivisions.csv.bz2 | Empty file |
| dgmAttributeCategories.csv.bz2 | Categories of dogma attributes categoryID - defined ID for this table |
| dgmAttributeTypes.csv.bz2 | Types of dogma attributes attributeID - defined ID for this table |
| dgmEffects.csv.bz2 | The effects of dogma attributes effectID - defined ID for this table |
| dgmExpressions.csv.bz2 | Empty file |
| dgmTypeAttributes.csv.bz2 | Assigns dogma attribute types to item type typeID - the ID of the item. Correponds to typeID in invTypes |
| dgmTypeEffects.csv.bz2 | Assigns dogma effects to item types typeID - the ID of the item. Corresponds to typeID in invTypes |
| eveGraphics.csv.bz2 | Graphics resources graphicID - defined ID for this table |
| eveIcons.csv.bz2 | Icon resources iconID - defined ID for this table |
| eveUnits.csv.bz2 | Describes units used in dogma and item properties unitID - defined ID for this table |
| industryActivity.csv.bz2 | Describes industrial activities possible for each blueprint typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| industryActivityMaterials.csv.bz2 | Describes materials required to perform each industrial activity for blueprints typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| industryActivityProbabilities.csv.bz2 | Describes outcomes and probabilities of inventions typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| industryActivityProducts.csv.bz2 | Describes outcomes of industrial activities typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| industryActivityRaces.csv.bz2 | Empty file |
| industryActivitySkills.csv.bz2 | Describes skills required perform industrial activities typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| industryBlueprints.csv.bz2 | Describes the base maximum number of BPC runs typeID - the ID of the blueprint. Correponds to typeID in invTypes |
| invCategories.csv.bz2 | Item category IDs and names categoryID - defined ID for this table |
| invContrabandTypes.csv.bz2 | Describes items considered contraband for each faction factionID - the ID of factions. Corresponds to factionID in chrFactions |
| invControlTowerResourcePurposes.csv.bz2 | Empty file |
| invControlTowerResources.csv.bz2 | Describes control tower properties controlTowerTypeID - the ID of control towers. Corresponds to typeID in invTypes |
| invFlags.csv.bz2 | Flags describing various items flagID - defined ID for this table |
| invGroups.csv.bz2 | Describes inventory groups groupID - defined ID for this table |
| invItems.csv.bz2 | Describes items that have defined owners itemID - defined ID for this table |
| invMarketGroups.csv.bz2 | Defines the market group hierarchy marketGroupID - defined ID for this table |
| invMetaGroups.csv.bz2 | Description of meta groups metaGroupID - defined ID for this table |
| invMetaTypes.csv.bz2 | Assigns item types to meta groups typeID - the ID of the item type. Corresponds to typeID in invTypes |
| invNames.csv.bz2 | Describes unique items itemID - defined ID for this table |
| invPositions.csv.bz2 | Describes position of unique items itemID - ID of unique items. Corresponds to itemID in invNames |
| invTraits.csv.bz2 | Describes traites ("bonuses") of ships traitID - defined ID for this table |
| invTypeMaterials.csv.bz2 | Describes materials obtained from reprocessing of items typeID - the ID of the item to reprocess. Corresponds to typeID in invTypes |
| invTypeReactions.csv.bz2 | Empty file |
| invTypes.csv.bz2 | Defines item types typeID - defined ID for this table |
| invTypes-nodescription.csv.bz2 | Table invTypes excluding description |
| invUniqueNames.csv.bz2 | Describes unique names itemID - defined ID for this table |
| invVolumes.csv.bz2 | Describes volumes of packaged ships
typeID - the ID of the ship. Corresponds to typeID in invTypes |
| mapCelestialGraphics.csv.bz2 | Describes graphics settings of celestials celestialID - ID of celestials. Corresponds to celestialID in mapDenormalize |
| mapCelestialStatistics.csv.bz2 | Describes physical parameters of celestials celestialID - ID of celestials. Corresponds to celestialID in mapDenormalize |
| mapConstellationJumps.csv.bz2 | Describes gate jumps between constellations fromRegionID - ID of the starting region. Corresponds to regionID in mapRegions |
| mapConstellations.csv.bz2 | Describes constellations regionID - ID of the costellation's region. Corresponds to regionID in mapRegions and itemID in mapDenormalize |
| mapDenormalize.csv.bz2 | Describes celestials itemID - defined ID for this table |
| mapJumps.csv.bz2 | Describes gate jumps between solar systems stargateID - the ID of the starting solar system. Corresponds to solarSystemID in mapSolarSystems |
| mapLandmarks.csv.bz2 | Empty file |
| mapLocationWormholeClasses.csv.bz2 | Each location's wormholeClassID locationID - ID of the location. Corresponds to itemID in mapDenormalize |
| mapRegionJumps.csv.bz2 | Describes gate jumps between regions fromRegionID - the ID of the starting region. Corresponds to regionID in mapRegions |
| mapRegions.csv.bz2 | Describes regions regionID - ID of the region. Corresponds to itemID in mapDenormalize |
| mapSolarSystemJumps.csv.bz2 | Describes gate jumps between solar systems fromRegionID - ID of the starting region. Corresponds to regionID in mapRegions |
| mapSolarSystems.csv.bz2 | Describes solar systems regionID - ID of the solar system's region. Corresponds to regionID in mapRegions and itemID in mapDenormalize |
| mapUniverse.csv.bz2 | Defines the K- and J-space unverses universeID - ID of the universe. Corresponds to typeID in invTypes |
| planetSchematics.csv.bz2 | Describes Planetary Industry schematics schematicID - defined ID for this table |
| planetSchematicsPinMap.csv.bz2 | Associates pin types with schematics schematicID - ID of the schematic. Corresponds to schematicID in planetSchematics |
| planetSchematicsTypeMap.csv.bz2 | Describes schematic productivity schematicID - ID of the schematic. Corresponds to schematicID in planetSchematics |
| ramActivities.csv.bz2 | Describes types of industrial activities activityID - defined ID for this table |
| ramAssemblyLineStations.csv.bz2 | Describes industry activities available at NPC stations stationID - ID of NPC stations. Corresponds to stationID in staStations |
| ramAssemblyLineTypeDetailPerCategory.csv.bz2 | Describes time, material, and cost bonuses at NPC station types for each inventory category assemblyLineTypeID - ID of assembly line type (station service type). Corresponds to assemblyLineTypeID in ramAssemblyLineTypes |
| ramAssemblyLineTypeDetailPerGroup.csv.bz2 | Describes time, material, and cost bonuses at NPC station types for each inventory group assemblyLineTypeID - ID of assembly line type (station service type). Corresponds to assemblyLineTypeID in ramAssemblyLineTypes |
| ramAssemblyLineTypes.csv.bz2 | Describes installation services assemblyLineTypeID - defined ID for this table |
| ramInstallationTypeContents.csv.bz2 | Assigns station services to installation types installationTypeID - ID of installation types. Corresponds to typeID in invTypes |
| skinLicense.csv.bz2 | Assigns licence types to skin types licenseTypeID - ID of licence types. Corresponds to typeID in invTypes |
| skinMaterials.csv.bz2 | skinMaterialID - |
| skins.csv.bz2 | skinID - |
| skinShip.csv.bz2 | Assigns skin types to ships skinID - ID of skins. Corresponds to skinID in skins |
| solarsystemprecise.csv.bz2 | Empty file |
| staOperations.csv.bz2 | activityID - |
| staOperationServices.csv.bz2 | operationID - |
| staServices.csv.bz2 | Description of NPC station services serviceID - defined ID for this table |
| staStations.csv.bz2 | Describes NPC stations stationID - ID of the station. Corresponds to itemID in mapDenormalize |
| staStationTypes.csv.bz2 | stationTypeID - |
| translationTables.csv.bz2 | Empty file |
| trnTranslationColumns.csv.bz2 | tcGroupID - |
| trnTranslationLanguages.csv.bz2 | numericLanguageID - |
| trnTranslations.csv.bz2 | |
| warCombatZones.csv.bz2 | combatZoneID - |
| warCombatZoneSystems.csv.bz2 | solarSystemID - |
External links
- CodeCademy
- Corey Schafer
- EVE Online Discord Dev Chat 3rd-party-dev-and-esi - I read most messages sent here, if you need help that StackOverflow can't provide, here is your best bet.