This simply means the scraper was not able to find something there. Basically the way it works is that we know through convention what the serial numbers are going to be; so we can generate all the possible
VIN combinations; we then beat the crap out of kia's website looking for those
vin combinations. If we find one, we download the pdf, extract the info.
Sometimes it's just a fluke, and the scraper will get temporarily banned and skip over that spot; sometimes for some reason the same serial numbers just never get a
vin. i have thought a lot about why, but I don't know. my guess is that something goes awry in production, or, they are only in the database until some transaction happens and then they get removed.
i have been thinking also about how to expose the UI for this; but you can manually trigger a scrape attempt for a given serial by visiting this url:
https://failcat-rust.vteng.io/scrape/<some_six_digit_serial>
the scraper itself is just a simple script, if you want to contribute to the cause
Code:
sleep 1.5
UP_FROM=412780
i=$(($UP_FROM+1))
echo "Attempting to scrape $i";
while time curl https://failcat-rust.vteng.io/scrape/$i;
sleep 6 && echo;
do echo;
i=$(($i+1))
echo "Attempting to scrape $i";
done
so, for example, if you wanted to trigger the scraper for 413852 you could visit
https://failcat-rust.vteng.io/scrape/413852 and that will
- generate all possible vins for that serial
- try each in a series to see if a pdf is available
- save to database if it is
if you see a number response, it was successful. if you see 'null' it means we generated all possible vins successfully, and tried all the vins successfully, but none were available.
Did you find an answer to this? Mine is one that's missing in a string too
View attachment 30804
