Difference between revisions of "Static Data Export"
Biwako Acami (talk | contribs) (→Opening a file from the SDE: Changed title) |
m (Change heading levels to avoid using lvl1 in the article content. Change external links to interwiki links.) |
||
(11 intermediate revisions by 7 users not shown) | |||
Line 1: | Line 1: | ||
− | CCP | + | CCP provides developers a series of static files, known as the Static Data Export(SDE), which contains static data from the Tranquility server. The SDE is currently exported as .yaml and .static files, however, different and easier to use formats(mentioned below) are made by the community. The SDE can be found at the EVE developers [https://developers.eveonline.com/resource/resources resource] page. All resources provided by CCP are subject to the [https://developers.eveonline.com/resource/license-agreement license agreement] linked on the page. |
− | = Fuzzwork SDE Conversions = | + | == Fuzzwork SDE Conversions == |
− | To aide fellow developers and players in consuming this data without having to extract or convert from | + | To aide fellow developers and players in consuming this data without having to extract or convert from YAML or static format every time Steve Ronuken has hosted conversions in PostgreSQL, SQLite, MySQL, MSSQL, and CSV formats. They can be found at [https://www.fuzzwork.co.uk Fuzzwork] under SDE or more directly at [https://www.fuzzwork.co.uk/dump www.fuzzwork.co.uk/dump]. Individual table data can be found in [[:wikipedia:Comma-separated values|CSV]] and [[:wikipedia:SQL|SQL]] (MySQL) formats here [https://www.fuzzwork.co.uk/dump/latest www.fuzzwork.co.uk/dump/latest]. |
− | == Opening a file from Fuzzwork SDE Conversions == | + | |
− | To extract data from compressed files, use [https://ss64.com/bash/tar.html tar] for <code>.tar.bz2</code> files or [https://ss64.com/osx/bzip.html bunzip2] for <code>.bz2</code> files in | + | === Opening a file from Fuzzwork SDE Conversions === |
− | Once extracted the files can be opened in the appropriate application that supports the file format. | + | To extract data from compressed files, use [https://ss64.com/bash/tar.html tar] for <code>.tar.bz2</code> files or [https://ss64.com/osx/bzip.html bunzip2] for <code>.bz2</code> files in Linux, and something like [https://www.7-zip.org/ 7-zip] for either format in windows. Once extracted the files can be opened in the appropriate application that supports the file format. |
+ | |||
+ | == How does the ESI fit into all this? == | ||
+ | The [[EVE Swagger Interface]] intends to have endpoints to account for all of the SDE, currently, it's not quite there yet, and the SDE is needed for many projects. The progress can be tracked [https://github.com/esi/esi-issues/issues/1103 here]. | ||
+ | |||
+ | == How to implement the original SDE into your application == | ||
+ | |||
+ | === Step 0: Preface === | ||
+ | This is not a guide to Python or programming, if you don't know how to program, there are tons of resources to learn to code, there are a few of my favorites in the External Links section down below, if your having issues, contact me at [[User:Asgore Astroficus|Asgore Astroficus]] ([[User talk:Asgore Astroficus|talk]]). | ||
+ | |||
+ | === Step 1: Opening the file === | ||
+ | Firstly, find the file containing the data you need, for this example, I'm going to use fsd/blueprints.yaml. This file contains all the information required to build an industry cost predictor. Next, open the file using your desired programming language, I'm going to use python, so to open this I'll run ''with open("fsd/blueprints.yaml", "r") as f:''. The file is open! To the next step! | ||
+ | |||
+ | === Step 2: Reading the file === | ||
+ | ==== Step 2.1: Installing PyYAML ==== | ||
+ | Assuming you're using pip, this should be as simple as ''pip install pyyaml''. If you having issues, contact me at [[User:Asgore Astroficus|Asgore Astroficus]] ([[User talk:Asgore Astroficus|talk]]) or have a look at [https://stackabuse.com/reading-and-writing-yaml-to-a-file-in-python/ this article]. | ||
+ | |||
+ | ==== Step 2.2: Interpreting the file ==== | ||
+ | Now that PyYAML is installed, we need to use it to parse our file, we have already opened it, so it should just be a matter of parsing it with PyYAML, which can be done by ''blueprints = yaml.load(f, Loader=yaml.FullLoader)'', remember to have ''import yaml'' at the top of your .py file. Hopefully now if you print out the blueprints variable, your console will be filled with sweet, sweet data and the taste of success. The data will be a dictionary, and the rest from here should be relatively simple, it should just be navigating through the data as if it was a dictionary. | ||
+ | |||
+ | === Step 3: Postface === | ||
+ | I hope this short guide helped you somewhat get into the SDE, I might write one for the Fuzzwork conversions soon. If you have any issues, pop me a message here [[User:Asgore Astroficus|Asgore Astroficus]] ([[User talk:Asgore Astroficus|talk]]). | ||
== What each table contains == | == What each table contains == | ||
Line 14: | Line 35: | ||
| [https://www.fuzzwork.co.uk/dump/latest/agtAgents.csv.bz2 agtAgents.csv.bz2]|| Contains information regarding agents such as location, corporation, type, level, and locator (boolean) | | [https://www.fuzzwork.co.uk/dump/latest/agtAgents.csv.bz2 agtAgents.csv.bz2]|| Contains information regarding agents such as location, corporation, type, level, and locator (boolean) | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/agtAgentTypes.csv.bz2 agtAgentTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/agtAgentTypes.csv.bz2 agtAgentTypes.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/agtResearchAgents.csv.bz2 agtResearchAgents.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/agtResearchAgents.csv.bz2 agtResearchAgents.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/certCerts.csv.bz2 certCerts.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/certCerts.csv.bz2 certCerts.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/certMasteries.csv.bz2 certMasteries.csv.bz2]|| What ships have what certificates. In this case, typeID is the ship, and certID is the certificate. See certSkills | | [https://www.fuzzwork.co.uk/dump/latest/certMasteries.csv.bz2 certMasteries.csv.bz2]|| What ships have what certificates. In this case, typeID is the ship, and certID is the certificate. See certSkills | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/certSkills.csv.bz2 certSkills.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/certSkills.csv.bz2 certSkills.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/chrAncestries.csv.bz2 chrAncestries.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/chrAncestries.csv.bz2 chrAncestries.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/chrAttributes.csv.bz2 chrAttributes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/chrAttributes.csv.bz2 chrAttributes.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/chrBloodlines.csv.bz2 chrBloodlines.csv.bz2]|| Contains information on what race each bloodline belongs to, descriptions. | | [https://www.fuzzwork.co.uk/dump/latest/chrBloodlines.csv.bz2 chrBloodlines.csv.bz2]|| Contains information on what race each bloodline belongs to, descriptions. | ||
Line 36: | Line 57: | ||
| [https://www.fuzzwork.co.uk/dump/latest/crpActivities.csv.bz2 crpActivities.csv.bz2]|| IDK, maybe names of NPC sturcture groups? | | [https://www.fuzzwork.co.uk/dump/latest/crpActivities.csv.bz2 crpActivities.csv.bz2]|| IDK, maybe names of NPC sturcture groups? | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationDivisions.csv.bz2 crpNPCCorporationDivisions.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationDivisions.csv.bz2 crpNPCCorporationDivisions.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationResearchFields.csv.bz2 crpNPCCorporationResearchFields.csv.bz2]|| Unknown | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationResearchFields.csv.bz2 crpNPCCorporationResearchFields.csv.bz2]|| Unknown | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporations.csv.bz2 crpNPCCorporations.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporations.csv.bz2 crpNPCCorporations.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationTrades.csv.bz2 crpNPCCorporationTrades.csv.bz2]|| Information on what each NPC corp sells. | | [https://www.fuzzwork.co.uk/dump/latest/crpNPCCorporationTrades.csv.bz2 crpNPCCorporationTrades.csv.bz2]|| Information on what each NPC corp sells. | ||
Line 48: | Line 69: | ||
| [https://www.fuzzwork.co.uk/dump/latest/dgmAttributeCategories.csv.bz2 dgmAttributeCategories.csv.bz2]|| What each categoryID is, and a categoryName | | [https://www.fuzzwork.co.uk/dump/latest/dgmAttributeCategories.csv.bz2 dgmAttributeCategories.csv.bz2]|| What each categoryID is, and a categoryName | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/dgmAttributeTypes.csv.bz2 dgmAttributeTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/dgmAttributeTypes.csv.bz2 dgmAttributeTypes.csv.bz2]|| Defines attributes of types (like CPU Load, Powergrid Usage, EM damage) |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/dgmEffects.csv.bz2 dgmEffects.csv.bz2]|| Information regarding effects, such as launching missiles, scaling damage, and the effects skills have. | | [https://www.fuzzwork.co.uk/dump/latest/dgmEffects.csv.bz2 dgmEffects.csv.bz2]|| Information regarding effects, such as launching missiles, scaling damage, and the effects skills have. | ||
Line 54: | Line 75: | ||
| [https://www.fuzzwork.co.uk/dump/latest/dgmExpressions.csv.bz2 dgmExpressions.csv.bz2]|| Information regarding expressions | | [https://www.fuzzwork.co.uk/dump/latest/dgmExpressions.csv.bz2 dgmExpressions.csv.bz2]|| Information regarding expressions | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/dgmTypeAttributes.csv.bz2 dgmTypeAttributes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/dgmTypeAttributes.csv.bz2 dgmTypeAttributes.csv.bz2]|| Provides values for the attributes of a type (how much EM damage does the Mjolnir Cruise Missile do) See dgmAttributeTypes and |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/dgmTypeEffects.csv.bz2 dgmTypeEffects.csv.bz2]|| effectID that each typID had, see also dgmEffects.csv.bz2 | | [https://www.fuzzwork.co.uk/dump/latest/dgmTypeEffects.csv.bz2 dgmTypeEffects.csv.bz2]|| effectID that each typID had, see also dgmEffects.csv.bz2 | ||
Line 60: | Line 81: | ||
| [https://www.fuzzwork.co.uk/dump/latest/eveGraphics.csv.bz2 eveGraphics.csv.bz2]|| Information regarding grapics such as graphicID, graphic location, race, and description. | | [https://www.fuzzwork.co.uk/dump/latest/eveGraphics.csv.bz2 eveGraphics.csv.bz2]|| Information regarding grapics such as graphicID, graphic location, race, and description. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/eveIcons.csv.bz2 eveIcons.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/eveIcons.csv.bz2 eveIcons.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/eveUnits.csv.bz2 eveUnits.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/eveUnits.csv.bz2 eveUnits.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivity.csv.bz2 industryActivity.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivity.csv.bz2 industryActivity.csv.bz2]|| Contains the number of seconds it takes to create a blueprint product |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityMaterials.csv.bz2 industryActivityMaterials.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityMaterials.csv.bz2 industryActivityMaterials.csv.bz2]|| Contains the materials required to create a blueprint product |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityProbabilities.csv.bz2 industryActivityProbabilities.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityProbabilities.csv.bz2 industryActivityProbabilities.csv.bz2]|| Contains the outcome and the probability of an invention |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityProducts.csv.bz2 industryActivityProducts.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityProducts.csv.bz2 industryActivityProducts.csv.bz2]|| Shows what processes create what (such as what products a blueprint creates) |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityRaces.csv.bz2 industryActivityRaces.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivityRaces.csv.bz2 industryActivityRaces.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryActivitySkills.csv.bz2 industryActivitySkills.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryActivitySkills.csv.bz2 industryActivitySkills.csv.bz2]|| Contains information about the skills required to manufacture any blueprint. |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/industryBlueprints.csv.bz2 industryBlueprints.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/industryBlueprints.csv.bz2 industryBlueprints.csv.bz2]|| Contains the base maximum amount of BPC runs |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/invCategories.csv.bz2 invCategories.csv.bz2]|| Category IDs and category names | | [https://www.fuzzwork.co.uk/dump/latest/invCategories.csv.bz2 invCategories.csv.bz2]|| Category IDs and category names | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invContrabandTypes.csv.bz2 invContrabandTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invContrabandTypes.csv.bz2 invContrabandTypes.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invControlTowerResourcePurposes.csv.bz2 invControlTowerResourcePurposes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invControlTowerResourcePurposes.csv.bz2 invControlTowerResourcePurposes.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invControlTowerResources.csv.bz2 invControlTowerResources.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invControlTowerResources.csv.bz2 invControlTowerResources.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invFlags.csv.bz2 invFlags.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invFlags.csv.bz2 invFlags.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/invGroups.csv.bz2 invGroups.csv.bz2]|| Contains information on inventory groups, such as groupID, categoryID, and other metadata | | [https://www.fuzzwork.co.uk/dump/latest/invGroups.csv.bz2 invGroups.csv.bz2]|| Contains information on inventory groups, such as groupID, categoryID, and other metadata | ||
Line 92: | Line 113: | ||
| [https://www.fuzzwork.co.uk/dump/latest/invItems.csv.bz2 invItems.csv.bz2]|| Information about itemID and typeID, and some others | | [https://www.fuzzwork.co.uk/dump/latest/invItems.csv.bz2 invItems.csv.bz2]|| Information about itemID and typeID, and some others | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invMarketGroups.csv.bz2 invMarketGroups.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invMarketGroups.csv.bz2 invMarketGroups.csv.bz2]|| Defines the market group hierarchy |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/invMetaGroups.csv.bz2 invMetaGroups.csv.bz2]|| A small file about meta group ids, such as Tech I, Tech II, Faction, Officer, etc | | [https://www.fuzzwork.co.uk/dump/latest/invMetaGroups.csv.bz2 invMetaGroups.csv.bz2]|| A small file about meta group ids, such as Tech I, Tech II, Faction, Officer, etc | ||
Line 100: | Line 121: | ||
| [https://www.fuzzwork.co.uk/dump/latest/invNames.csv.bz2 invNames.csv.bz2]|| itemID for everything. Not that itemID is not typeID. | | [https://www.fuzzwork.co.uk/dump/latest/invNames.csv.bz2 invNames.csv.bz2]|| itemID for everything. Not that itemID is not typeID. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invPositions.csv.bz2 invPositions.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invPositions.csv.bz2 invPositions.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invTraits.csv.bz2 invTraits.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invTraits.csv.bz2 invTraits.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invTypeMaterials.csv.bz2 invTypeMaterials.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invTypeMaterials.csv.bz2 invTypeMaterials.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invTypeReactions.csv.bz2 invTypeReactions.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invTypeReactions.csv.bz2 invTypeReactions.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invTypes.csv.bz2 invTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invTypes.csv.bz2 invTypes.csv.bz2]|| Defines the types in the game like market items and celestial objects. Contains typeID, name, description, marketGroupID, ... |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invUniqueNames.csv.bz2 invUniqueNames.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invUniqueNames.csv.bz2 invUniqueNames.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/invVolumes.csv.bz2 invVolumes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/invVolumes.csv.bz2 invVolumes.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapCelestialStatistics.csv.bz2 mapCelestialStatistics.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapCelestialStatistics.csv.bz2 mapCelestialStatistics.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapConstellationJumps.csv.bz2 mapConstellationJumps.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapConstellationJumps.csv.bz2 mapConstellationJumps.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/mapConstellations.csv.bz2 mapConstellations.csv.bz2]|| Constellation position, faction, name, region, name, and radius. | | [https://www.fuzzwork.co.uk/dump/latest/mapConstellations.csv.bz2 mapConstellations.csv.bz2]|| Constellation position, faction, name, region, name, and radius. | ||
Line 122: | Line 143: | ||
| [https://www.fuzzwork.co.uk/dump/latest/mapDenormalize.csv.bz2 mapDenormalize.csv.bz2]|| Position, radius, and security status of moons, planets, solar systems, all the way to regions. As such, a very large CSV with more than half a million lines. | | [https://www.fuzzwork.co.uk/dump/latest/mapDenormalize.csv.bz2 mapDenormalize.csv.bz2]|| Position, radius, and security status of moons, planets, solar systems, all the way to regions. As such, a very large CSV with more than half a million lines. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapJumps.csv.bz2 mapJumps.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapJumps.csv.bz2 mapJumps.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapLandmarks.csv.bz2 mapLandmarks.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapLandmarks.csv.bz2 mapLandmarks.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapLocationScenes.csv.bz2 mapLocationScenes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapLocationScenes.csv.bz2 mapLocationScenes.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/mapLocationWormholeClasses.csv.bz2 mapLocationWormholeClasses.csv.bz2]|| Each location's wormholeClassID | | [https://www.fuzzwork.co.uk/dump/latest/mapLocationWormholeClasses.csv.bz2 mapLocationWormholeClasses.csv.bz2]|| Each location's wormholeClassID | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapRegionJumps.csv.bz2 mapRegionJumps.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapRegionJumps.csv.bz2 mapRegionJumps.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapRegions.csv.bz2 mapRegions.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapRegions.csv.bz2 mapRegions.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapSolarSystemJumps.csv.bz2 mapSolarSystemJumps.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapSolarSystemJumps.csv.bz2 mapSolarSystemJumps.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/mapSolarSystems.csv.bz2 mapSolarSystems.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/mapSolarSystems.csv.bz2 mapSolarSystems.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/mapUniverse.csv.bz2 mapUniverse.csv.bz2]|| Possibly to differentiate between K-space and J-space? (speculation) | | [https://www.fuzzwork.co.uk/dump/latest/mapUniverse.csv.bz2 mapUniverse.csv.bz2]|| Possibly to differentiate between K-space and J-space? (speculation) | ||
Line 142: | Line 163: | ||
| [https://www.fuzzwork.co.uk/dump/latest/planetSchematics.csv.bz2 planetSchematics.csv.bz2]|| schematicID, name, and time it takes to complete that schematic. | | [https://www.fuzzwork.co.uk/dump/latest/planetSchematics.csv.bz2 planetSchematics.csv.bz2]|| schematicID, name, and time it takes to complete that schematic. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/planetSchematicsPinMap.csv.bz2 planetSchematicsPinMap.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/planetSchematicsPinMap.csv.bz2 planetSchematicsPinMap.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/planetSchematicsTypeMap.csv.bz2 planetSchematicsTypeMap.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/planetSchematicsTypeMap.csv.bz2 planetSchematicsTypeMap.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/ramActivities.csv.bz2 ramActivities.csv.bz2]|| Describes what each R.A.M. does. See [[Manufacturing]] | | [https://www.fuzzwork.co.uk/dump/latest/ramActivities.csv.bz2 ramActivities.csv.bz2]|| Describes what each R.A.M. does. See [[Manufacturing]] | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineStations.csv.bz2 ramAssemblyLineStations.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineStations.csv.bz2 ramAssemblyLineStations.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypeDetailPerCategory.csv.bz2 ramAssemblyLineTypeDetailPerCategory.csv.bz2]|| Information on R.A.M. | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypeDetailPerCategory.csv.bz2 ramAssemblyLineTypeDetailPerCategory.csv.bz2]|| Information on R.A.M. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypeDetailPerGroup.csv.bz2 ramAssemblyLineTypeDetailPerGroup.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypeDetailPerGroup.csv.bz2 ramAssemblyLineTypeDetailPerGroup.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypes.csv.bz2 ramAssemblyLineTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/ramAssemblyLineTypes.csv.bz2 ramAssemblyLineTypes.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/ramInstallationTypeContents.csv.bz2 ramInstallationTypeContents.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/ramInstallationTypeContents.csv.bz2 ramInstallationTypeContents.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/skinLicense.csv.bz2 skinLicense.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/skinLicense.csv.bz2 skinLicense.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/skinMaterials.csv.bz2 skinMaterials.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/skinMaterials.csv.bz2 skinMaterials.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/skins.csv.bz2 skins.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/skins.csv.bz2 skins.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/skinShip.csv.bz2 skinShip.csv.bz2]|| What skins can be used on what ships | | [https://www.fuzzwork.co.uk/dump/latest/skinShip.csv.bz2 skinShip.csv.bz2]|| What skins can be used on what ships | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/staOperations.csv.bz2 staOperations.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/staOperations.csv.bz2 staOperations.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/staOperationServices.csv.bz2 staOperationServices.csv.bz2]|| Information about what station types offer what services (speculation) | | [https://www.fuzzwork.co.uk/dump/latest/staOperationServices.csv.bz2 staOperationServices.csv.bz2]|| Information about what station types offer what services (speculation) | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/staServices.csv.bz2 staServices.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/staServices.csv.bz2 staServices.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/staStations.csv.bz2 staStations.csv.bz2]|| Information about stations, their security level, location, name, reprocessing level, and much more. | | [https://www.fuzzwork.co.uk/dump/latest/staStations.csv.bz2 staStations.csv.bz2]|| Information about stations, their security level, location, name, reprocessing level, and much more. | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/staStationTypes.csv.bz2 staStationTypes.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/staStationTypes.csv.bz2 staStationTypes.csv.bz2]|| |
|- | |- | ||
| [https://www.fuzzwork.co.uk/dump/latest/translationTables.csv.bz2 translationTables.csv.bz2]|| Appears to be empty | | [https://www.fuzzwork.co.uk/dump/latest/translationTables.csv.bz2 translationTables.csv.bz2]|| Appears to be empty | ||
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslationColumns.csv.bz2 trnTranslationColumns.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslationColumns.csv.bz2 trnTranslationColumns.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslationLanguages.csv.bz2 trnTranslationLanguages.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslationLanguages.csv.bz2 trnTranslationLanguages.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslations.csv.bz2 trnTranslations.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/trnTranslations.csv.bz2 trnTranslations.csv.bz2]|| |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/warCombatZones.csv.bz2 warCombatZones.csv.bz2]|| A small file that updates regularly. | + | | [https://www.fuzzwork.co.uk/dump/latest/warCombatZones.csv.bz2 warCombatZones.csv.bz2]|| A small file that updates regularly. It contains information about combat zones (FW I think?) |
|- | |- | ||
− | | [https://www.fuzzwork.co.uk/dump/latest/warCombatZoneSystems.csv.bz2 warCombatZoneSystems.csv.bz2]|| | + | | [https://www.fuzzwork.co.uk/dump/latest/warCombatZoneSystems.csv.bz2 warCombatZoneSystems.csv.bz2]|| |
|} | |} | ||
== External links == | == External links == | ||
− | * [https:// | + | * [https://codecademy.com CodeCademy] |
+ | * [https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g Corey Schafer] | ||
+ | * [https://discord.gg/csvQt9 EVE Online Discord Dev Chat] - I read most messages sent here, if you need help that StackOverflow can't provide, here is your best bet. | ||
[[Category:Applications]] | [[Category:Applications]] |
Latest revision as of 12:29, 13 July 2024
CCP provides developers a series of static files, known as the Static Data Export(SDE), which contains static data from the Tranquility server. The SDE is currently exported as .yaml and .static files, however, different and easier to use formats(mentioned below) are made by the community. The SDE can be found at the EVE developers resource page. All resources provided by CCP are subject to the license agreement linked on the page.
Fuzzwork SDE Conversions
To aide fellow developers and players in consuming this data without having to extract or convert from YAML or static format every time Steve Ronuken has hosted conversions in PostgreSQL, SQLite, MySQL, MSSQL, and CSV formats. They can be found at Fuzzwork under SDE or more directly at www.fuzzwork.co.uk/dump. Individual table data can be found in CSV and SQL (MySQL) formats here www.fuzzwork.co.uk/dump/latest.
Opening a file from Fuzzwork SDE Conversions
To extract data from compressed files, use tar for .tar.bz2
files or bunzip2 for .bz2
files in Linux, and something like 7-zip for either format in windows. Once extracted the files can be opened in the appropriate application that supports the file format.
How does the ESI fit into all this?
The EVE Swagger Interface intends to have endpoints to account for all of the SDE, currently, it's not quite there yet, and the SDE is needed for many projects. The progress can be tracked here.
How to implement the original SDE into your application
Step 0: Preface
This is not a guide to Python or programming, if you don't know how to program, there are tons of resources to learn to code, there are a few of my favorites in the External Links section down below, if your having issues, contact me at Asgore Astroficus (talk).
Step 1: Opening the file
Firstly, find the file containing the data you need, for this example, I'm going to use fsd/blueprints.yaml. This file contains all the information required to build an industry cost predictor. Next, open the file using your desired programming language, I'm going to use python, so to open this I'll run with open("fsd/blueprints.yaml", "r") as f:. The file is open! To the next step!
Step 2: Reading the file
Step 2.1: Installing PyYAML
Assuming you're using pip, this should be as simple as pip install pyyaml. If you having issues, contact me at Asgore Astroficus (talk) or have a look at this article.
Step 2.2: Interpreting the file
Now that PyYAML is installed, we need to use it to parse our file, we have already opened it, so it should just be a matter of parsing it with PyYAML, which can be done by blueprints = yaml.load(f, Loader=yaml.FullLoader), remember to have import yaml at the top of your .py file. Hopefully now if you print out the blueprints variable, your console will be filled with sweet, sweet data and the taste of success. The data will be a dictionary, and the rest from here should be relatively simple, it should just be navigating through the data as if it was a dictionary.
Step 3: Postface
I hope this short guide helped you somewhat get into the SDE, I might write one for the Fuzzwork conversions soon. If you have any issues, pop me a message here Asgore Astroficus (talk).
What each table contains
Table name | What it contains |
---|---|
agtAgents.csv.bz2 | Contains information regarding agents such as location, corporation, type, level, and locator (boolean) |
agtAgentTypes.csv.bz2 | |
agtResearchAgents.csv.bz2 | |
certCerts.csv.bz2 | |
certMasteries.csv.bz2 | What ships have what certificates. In this case, typeID is the ship, and certID is the certificate. See certSkills |
certSkills.csv.bz2 | |
chrAncestries.csv.bz2 | |
chrAttributes.csv.bz2 | |
chrBloodlines.csv.bz2 | Contains information on what race each bloodline belongs to, descriptions. |
chrFactions.csv.bz2 | Faction IDs, name, and description. Also what race each faction belongs to, and some other misc. information. |
chrRaces.csv.bz2 | raceID and raceName, iconID and a long and short description (side note, ORE is a race) |
crpActivities.csv.bz2 | IDK, maybe names of NPC sturcture groups? |
crpNPCCorporationDivisions.csv.bz2 | |
crpNPCCorporationResearchFields.csv.bz2 | Unknown |
crpNPCCorporations.csv.bz2 | |
crpNPCCorporationTrades.csv.bz2 | Information on what each NPC corp sells. |
crpNPCDivisions.csv.bz2 | Information on NPC divisions, mostly deprecated. |
dgmAttributeCategories.csv.bz2 | What each categoryID is, and a categoryName |
dgmAttributeTypes.csv.bz2 | Defines attributes of types (like CPU Load, Powergrid Usage, EM damage) |
dgmEffects.csv.bz2 | Information regarding effects, such as launching missiles, scaling damage, and the effects skills have. |
dgmExpressions.csv.bz2 | Information regarding expressions |
dgmTypeAttributes.csv.bz2 | Provides values for the attributes of a type (how much EM damage does the Mjolnir Cruise Missile do) See dgmAttributeTypes and |
dgmTypeEffects.csv.bz2 | effectID that each typID had, see also dgmEffects.csv.bz2 |
eveGraphics.csv.bz2 | Information regarding grapics such as graphicID, graphic location, race, and description. |
eveIcons.csv.bz2 | |
eveUnits.csv.bz2 | |
industryActivity.csv.bz2 | Contains the number of seconds it takes to create a blueprint product |
industryActivityMaterials.csv.bz2 | Contains the materials required to create a blueprint product |
industryActivityProbabilities.csv.bz2 | Contains the outcome and the probability of an invention |
industryActivityProducts.csv.bz2 | Shows what processes create what (such as what products a blueprint creates) |
industryActivityRaces.csv.bz2 | |
industryActivitySkills.csv.bz2 | Contains information about the skills required to manufacture any blueprint. |
industryBlueprints.csv.bz2 | Contains the base maximum amount of BPC runs |
invCategories.csv.bz2 | Category IDs and category names |
invContrabandTypes.csv.bz2 | |
invControlTowerResourcePurposes.csv.bz2 | |
invControlTowerResources.csv.bz2 | |
invFlags.csv.bz2 | |
invGroups.csv.bz2 | Contains information on inventory groups, such as groupID, categoryID, and other metadata |
invItems.csv.bz2 | Information about itemID and typeID, and some others |
invMarketGroups.csv.bz2 | Defines the market group hierarchy |
invMetaGroups.csv.bz2 | A small file about meta group ids, such as Tech I, Tech II, Faction, Officer, etc |
invMetaTypes.csv.bz2 | typeID, parentTypeID, metaGroupID |
invNames.csv.bz2 | itemID for everything. Not that itemID is not typeID. |
invPositions.csv.bz2 | |
invTraits.csv.bz2 | |
invTypeMaterials.csv.bz2 | |
invTypeReactions.csv.bz2 | |
invTypes.csv.bz2 | Defines the types in the game like market items and celestial objects. Contains typeID, name, description, marketGroupID, ... |
invUniqueNames.csv.bz2 | |
invVolumes.csv.bz2 | |
mapCelestialStatistics.csv.bz2 | |
mapConstellationJumps.csv.bz2 | |
mapConstellations.csv.bz2 | Constellation position, faction, name, region, name, and radius. |
mapDenormalize.csv.bz2 | Position, radius, and security status of moons, planets, solar systems, all the way to regions. As such, a very large CSV with more than half a million lines. |
mapJumps.csv.bz2 | |
mapLandmarks.csv.bz2 | |
mapLocationScenes.csv.bz2 | |
mapLocationWormholeClasses.csv.bz2 | Each location's wormholeClassID |
mapRegionJumps.csv.bz2 | |
mapRegions.csv.bz2 | |
mapSolarSystemJumps.csv.bz2 | |
mapSolarSystems.csv.bz2 | |
mapUniverse.csv.bz2 | Possibly to differentiate between K-space and J-space? (speculation) |
planetSchematics.csv.bz2 | schematicID, name, and time it takes to complete that schematic. |
planetSchematicsPinMap.csv.bz2 | |
planetSchematicsTypeMap.csv.bz2 | |
ramActivities.csv.bz2 | Describes what each R.A.M. does. See Manufacturing |
ramAssemblyLineStations.csv.bz2 | |
ramAssemblyLineTypeDetailPerCategory.csv.bz2 | Information on R.A.M. |
ramAssemblyLineTypeDetailPerGroup.csv.bz2 | |
ramAssemblyLineTypes.csv.bz2 | |
ramInstallationTypeContents.csv.bz2 | |
skinLicense.csv.bz2 | |
skinMaterials.csv.bz2 | |
skins.csv.bz2 | |
skinShip.csv.bz2 | What skins can be used on what ships |
staOperations.csv.bz2 | |
staOperationServices.csv.bz2 | Information about what station types offer what services (speculation) |
staServices.csv.bz2 | |
staStations.csv.bz2 | Information about stations, their security level, location, name, reprocessing level, and much more. |
staStationTypes.csv.bz2 | |
translationTables.csv.bz2 | Appears to be empty |
trnTranslationColumns.csv.bz2 | |
trnTranslationLanguages.csv.bz2 | |
trnTranslations.csv.bz2 | |
warCombatZones.csv.bz2 | A small file that updates regularly. It contains information about combat zones (FW I think?) |
warCombatZoneSystems.csv.bz2 |
External links
- CodeCademy
- Corey Schafer
- EVE Online Discord Dev Chat - I read most messages sent here, if you need help that StackOverflow can't provide, here is your best bet.