Archived Forum Post

Index of archived forum posts

Question:

SkipUnspidered Problem

Oct 22 '15 at 09:09

Hi, I have a program that find in the database the already crawled links, then I want to add it to the spidered list with SkipUnspidered function (I haven't find a better way) This function take the index of the URL in the Unspidered List and must put it in the Spidered List But that list remains empty

Here my code :
std::string temp;
std::string s = ""ur":["";
size_t begin = json.find(s);
size_t end = json.find(""]",begin);
bool b = true;
temp = json.substr(begin + s.size(), end - begin - s.size());
this->spider->AddUnspidered(temp.c_str());
this->spider->SkipUnspidered(0);
while (b)
{
begin = json.find(s, end);
if (begin != std::string::npos)
{
end = json.find(""]", begin);
temp = json.substr(begin + s.size(), end - begin - s.size());
this->spider->AddUnspidered(temp.c_str());
this->spider->SkipUnspidered(0);
}
else
{
b = false;
}
}