Data Formats Explained¶

Shapefiles, CSVs, Rasters, APIs — demystified in plain language.

Why Formats Matter¶

When you find a dataset, it usually comes in a specific file format. The format tells you:

What software can open it
What kind of data it contains (tables? maps? images?)
What skills you need to use it

This page explains the most common formats you will encounter in this book.

Non-Spatial Formats (Tables & Reports)¶

These formats contain rows and columns of data — no geographic component.

📊 CSV (Comma-Separated Values)¶

The most common, most useful format

A CSV is essentially a spreadsheet saved as plain text. Every row is a record (e.g., a district), every column is a field (e.g., population, literacy rate).

Pros	Cons
Opens in Excel, Google Sheets, Python, R	No geographic info (just names/codes)
Small file size	No styling or formulas
Works everywhere	Large files can be slow

How to open: Excel → File → Open → Browse to .csv file

Example in this book: Census population tables, MGNREGA work data, UDISE+ school data

📄 PDF / Reports¶

Human-readable, not machine-readable

Many government datasets come as PDF reports. These are great for reading but difficult to analyse automatically.

Tip: Look for the "Data" or "Tables" section on the same portal — often a downloadable Excel or CSV version exists alongside the PDF.

Example in this book: ASER Annual Reports, Forest Survey of India reports

📋 Excel / ODS¶

Spreadsheet with multiple sheets

Similar to CSV but with multiple tabs, formulas, and formatting. Government portals often provide both Excel and CSV.

How to open: Microsoft Excel, LibreOffice Calc, Google Sheets

Spatial / Geographic Formats¶

These formats store location information — points, lines, polygons, or images tied to real-world coordinates.

📁 Shapefile (.shp)¶

The most common GIS vector format

A Shapefile is not one file — it's a bundle of at least 3 files:

district_boundary/
├── districts.shp   → The actual geometry (shapes)
├── districts.dbf   → The attribute table (data about each shape)
├── districts.prj   → The projection (coordinate system)
└── districts.shx   → Index file

Keep All Files Together

Never move just the .shp file — always keep the entire folder. QGIS and other GIS software need all files to open a Shapefile correctly.

How to open: QGIS (free), ArcGIS, Python (geopandas)

Example in this book: District boundaries, PMGSY road networks, village boundaries

🗺️ GeoJSON (.geojson)¶

The modern, web-friendly vector format

GeoJSON stores the same information as Shapefiles but in a single file, readable by any text editor. It's widely used in web maps.

How to open: QGIS, any text editor, JavaScript/Python

When to use: Web development, sharing data online, modern GIS workflows

🖼️ GeoTIFF (.tif / .tiff)¶

Raster image with geographic coordinates

A GeoTIFF looks like a photograph but it's attached to real-world coordinates. Each pixel has a geographic location.

Examples: - Satellite images (Landsat, Sentinel) - Elevation models (DEM) - Land use maps - Rainfall grids

How to open: QGIS, Python (rasterio, rioxarray)

🌐 NetCDF (.nc)¶

Scientific data for climate and weather

NetCDF (Network Common Data Form) is used for multi-dimensional data like rainfall over time, temperature grids, and ocean data.

Example in this book: IMD gridded rainfall data (daily/monthly, 0.25° grid across India)

How to open: Python (xarray, netCDF4), QGIS with NetCDF plugin

Web Service Formats¶

These are not files you download — instead, you connect to a live server.

🌍 WMS — Web Map Service¶

A map that loads live from a server

WMS delivers pre-rendered map tiles (images) from a server. You can view them in QGIS or embed them in a web map, but you cannot query individual features.

Use case: Load Bhuvan satellite imagery as a background layer in QGIS.

How to use: QGIS → Layer → Add Layer → Add WMS/WMTS Layer → Enter URL

🔷 WFS — Web Feature Service¶

Vector data delivered live from a server

WFS delivers actual vector features (geometries + attributes) that you can query, filter, and download.

Use case: Get all river polygons within a specific district from the Bhuvan server.

How to use: QGIS → Layer → Add Layer → Add WFS Layer → Enter URL

⚡ API (Application Programming Interface)¶

Programmatic access to data

An API lets software communicate with a data server. Instead of downloading a file, you send a request and get back data in JSON format.

Use case: Get the latest MGNREGA employment data for your district using a Python script.

How to use: Python requests library or JavaScript fetch()

Format Decision Guide¶

I need data for...
│
├── Excel analysis or charts?
│   → CSV or Excel (.xlsx)
│
├── Making a map in QGIS?
│   ├── Boundaries / roads / points? → Shapefile or GeoJSON
│   └── Satellite image or rainfall grid? → GeoTIFF or NetCDF
│
├── Web development / JavaScript?
│   → GeoJSON or API (JSON)
│
└── Python / R data science?
    ├── Table data? → CSV
    ├── Spatial vector? → GeoJSON (geopandas)
    ├── Raster? → GeoTIFF (rasterio)
    └── Climate/time-series? → NetCDF (xarray)

✏️ Exercise: Identify the Format¶

Exercise 0.2 — Match the Format

For each use case below, decide which format would be best:

You want to show the boundaries of India's states on a Google Maps-style web app.
You want to analyse district-wise literacy rates in Python using pandas.
You want to visualise monthly rainfall patterns across India for 10 years.
You want to add a satellite imagery background to your QGIS map.
You want to automatically fetch live data from a government portal every day.

Answers: GeoJSON | CSV | NetCDF | WMS/GeoTIFF | API

Next Chapter: Chapter 1 — Administrative Boundaries & Maps →