Archived Forum Post

Index of archived forum posts

Question:

Spider no longer crawls past first URL

Aug 31 '15 at 17:50

I'm using the 64-bit C# .NET 4.5.* version of the Spider and until 2 days ago it was working great. All of a sudden, however, the Spider started failing on the first URL of any page I gave it with the error that there were no un-spidered URLs remaining even though I know there are plenty. This is happening both on sites I've successfully spidered previously and on sites I've never touched.

The Spider is listed as "freeware" so I didn't think the 30-day license trial period would apply, but I did first start using it approximately 30 days ago so time-wise it would make sense. I've contacted Chilkat about this but haven't yet heard back.

Aside from the license, is there anything else that might cause the Spider to stop working? Error message is below.

Error Text: ChilkatLog:
CrawlNext:
    DllDate: Jun 23 2015
    ChilkatVersion: 9.5.0.51
    UnlockPrefix: NONE
    Username: [removed]
    Architecture: Little Endian; 64-bit
    Language: .NET 4.5 / x64
    VerboseLogging: 0
    _crawl:
      No unspidered URLs remaining.
    --_crawl
    Failed.
  --CrawlNext
--ChilkatLog
 Html: ChilkatLog:<blockquote>
CrawlNext:<blockquote>
DllDate: Jun 23 2015<br>
ChilkatVersion: 9.5.0.51<br>
UnlockPrefix: NONE<br>
Username: [removed]<br>
Architecture: Little Endian; 64-bit<br>
Language: .NET 4.5 / x64<br>
VerboseLogging: 0<br>
_crawl:<blockquote>
No unspidered URLs remaining.<br>
</blockquote>
Failed.<br>
</blockquote>
</blockquote>

Answer

Update: Deleting everything from the Spider's cache seems to have helped the problem. The Spider appears to be working again, but I'm a little disappointed that my solution carries such a heavy impact on the speed of my program since the cache had to be wiped. Is this a known/solved issue for anyone else?