AusCommodities
MapGrainsLivestockWoolDairyCarbonAbout

About & Methodology

AusCommodities Dashboard aggregates public data from 14+ sources to provide a real-time view of the Australian agricultural commodities market — covering livestock, dairy, grains, wool, carbon credits, forex, climate, and satellite vegetation indices.

Data Sources

All data is sourced from publicly available APIs, government datasets, and industry platforms. No paid data subscriptions are used.

SourceDataAPI / MethodUpdate FrequencyAccess
MLA Statistics APIEYCI, NTLI, NFS, NPCREST API (/report/5)DailyFree
MLA Statistics APIMeat Exports by destinationREST API (/report/1)DailyFree
MLA Statistics APIGlobal Cattle Prices (AU vs US)REST API (/report/7)DailyFree
Global Dairy TradeWMP, SMP, AMF, Butter, CheddarPublic S3 JSONTuesdays 6amFree
Yahoo FinanceWheat, Corn, Soybean + GrainCorpyfinance (Python)DailyFree
Yahoo FinanceAUD/USD exchange rateyfinance (Python)DailyFree
Yahoo FinanceKRBN, KEUA, CO2.L carbon ETFsyfinance (Python)DailyFree
wool.comEMI, MPG micron gradesWeb scrapingWed / Thu (auction days)Free
Open-Meteo27 weather stations — temp, rain, solarArchive REST APIDailyFree
NASA ORNL DAACNDVI for 26 production regionsMODIS REST APIWeekly (16-day satellite)Free
Beef/Sheep/Grain Central + othersIndustry news articlesRSS + Web scraping3x dailyFree
UN ComtradeChina beef import sourcesPreview REST APIWeekly (Monday)Free
AIP + World Bank + RBADiesel, RBA rate, Urea pricesExcel + CSV downloadWeekly (Monday)Free
Esri World ImagerySatellite imagery tilesTile proxy + Redis cacheOn demand (3-day TTL)Free

Methodology

State Coloring

State fill colors on the interactive map are derived from NDVI (Normalized Difference Vegetation Index) values. NASA MODIS 250m satellite data is fetched for each of the 26 production regions, then averaged by state. The green-to-brown color scale represents vegetation health — greener states indicate healthier pasture conditions.

Feed Cost Ratio

Calculated as EYCI (c/kg) divided by CBOT Wheat price (converted to AUD/t). A rising ratio indicates expanding cattle margins relative to feed costs; a falling ratio signals a feed cost squeeze. This is a widely-used profitability proxy in the Australian livestock industry.

52-Week Range Bars

Price cards display a 52-week range bar showing where the current price sits within its annual trading range. The bar is calculated from the minimum and maximum closing prices over the trailing 52 weeks of available data.

Market Pulse Sorting

The Market Pulse panel on the home page ranks commodities by absolute percentage change (descending). This surfaces the most volatile movers first, regardless of direction, giving a quick snapshot of where the action is.

Technology

Backend

Python FastAPI with SQLite for persistent storage, APScheduler for data fetching schedules, and Redis for caching satellite tiles and rate-limited responses.

Frontend

Next.js 16 (App Router) with TypeScript, Tailwind CSS, shadcn/ui components, react-simple-maps for the interactive Australia map, and Recharts for time series visualizations.

Deployment

Docker Compose orchestrating three services: Redis, FastAPI backend, and Next.js frontend. All data fetching runs server-side on scheduled intervals.

Known Limitations

  • Yahoo Finance rate limits can cause temporary data gaps for forex and carbon ETF prices. The backend implements retry logic, but extended outages may show stale data.
  • Wool auction data (EMI, MPG grades) accumulates over time from scraping. Only the current auction snapshot is available on each scrape — no historical backfill from wool.com.
  • WaterNSW API is unreliable and frequently returns errors. Water data may be unavailable or outdated.
  • ACCU (Australian Carbon Credit Unit) spot price is indicative only. No free real-time API exists for this market — the displayed value is a manual reference price.

Built as a portfolio project to demonstrate full-stack data engineering with real-time Australian market data.

Source Code→