Crawl Queue Manager

Active crawler scheduling queue, depth tracking, retry counters, and worker task states.