Metal Scraper Dashboard

Run scraper jobs and monitor status, logs, and recent manifests.

Bulk Browse Import FAST · 193k bands · ~30 min

Uses the Metal Archives browse JSON API (~400 requests, 4-7 s apart) to import name, country, genres, status for every band — no per-band page scraping. Run this first to seed the database, then use the detail scraper below to enrich records.

Enrich from DB DETAIL · bio · members · albums

Queries the DB for bands with no bio yet and enriches them one by one with full detail data (bio, members, albums, tracklists, social links, images). Runs at ~8-12s per band to stay polite.

Pipeline queues mb-fill · caa-covers · ma-band · wiki

One supervisor, four named queues (no AI). Backlog / throughput / ETA are per queue — not a single band counter. Matching is a separate prerequisite: python -m metal_scraper.matching.

Queue Status Backlog Throughput ETA 

    


    
    

      

        
Detail Scraper per-band page

        
Enriches bands with bio, members, albums, tracklists, social links.

        
        
        
        
        
        
        

          
          
        

        

          
          
        

        

      

      

        
Current Status

        
Loading...

      

    


    

      
Enrichment Log

      

    

    

      
Browse Import Log

      

    

    

      
Detail Scraper Log

      

    

    

      
Recent Runs

      

        
          

            Run
            Status
            Started
            Finished
            Counts