我正在 eBay 上抓取汽车图片和信息。这是link 。但是，如果我在 start_requests 函数中生成超过 2 个网页，则 scrapy 开始生成重复的项目，而在我爬行的页面中，没有重复的项目。下图展示了 scrapy 如何生成重复项:
4、6、8 号车是重复的。似乎 scrapy 在生成这些项目时已经出现故障。 但是，如果我在 start_request 函数中仅生成 1 或 2 个网页，则爬虫程序可以正常工作。 这是我的代码:
import scrapy
import urllib
from amazon.items import AmazonItem
class carSpider(scrapy.Spider):
imgcount = 1
name = "car"
allowed_domains = ["ebay.com"]
def start_requests(self):
yield scrapy.Request("http://www.ebay.com/sch/Cars-Trucks-/6001/i.html",self.parse)
for i in range(2,20):
yield scrapy.Request("http://www.ebay.com/sch/Cars-Trucks-/6001/i.html?_pgn="+str(i)+"&_skc=50&rt=nc",self.parse)
def parse(self,response):
namelist = response.xpath('//a[@class="vip"]/@title').extract()
htmllist = response.xpath('//a[@class="vip"]/@href').extract()
#imglist = response.xpath('//a[@class="img imgWr2"]/img/@src').extract()
listlength = len(namelist)
for i in range(0,listlength):
item = AmazonItem()
item['Name'] = namelist[i]
item['Source'] = htmllist[i]
imgnm = htmllist[i].split('g:')[-1].split('&')[0]
imgurl = 'http://i.ebayimg.com/images/g/'+imgnm+'/s-l500.jpg'
urllib.urlretrieve(imgurl,"~Desktop/carImages/"+str(carSpider.imgcount)+".jpg")
item['Path'] = "~/Desktop/carImages/"+str(carSpider.imgcount)+".jpg"
carSpider.imgcount = carSpider.imgcount + 1
yield item
有人有什么建议吗？
最佳答案
您生成的 URL 仅在 HTTP GET 参数中不同 - 这些 URL 在 Scrapy 中被视为重复。通过 dont_filter=True 禁用过滤:
yield scrapy.Request("http://www.ebay.com/sch/Cars-Trucks-/6001/i.html?_pgn="+str(i)+"&_skc=50&rt=nc",
callback=self.parse,
dont_filter=True)
仅供引用，这是我用来测试解决方案的一个简单修改的蜘蛛:
import scrapy
class AmazonItem(scrapy.Item):
Name = scrapy.Field()
Source = scrapy.Field()
class carSpider(scrapy.Spider):
imgcount = 1
name = "car"
allowed_domains = ["ebay.com"]
def start_requests(self):
yield scrapy.Request("http://www.ebay.com/sch/Cars-Trucks-/6001/i.html", self.parse)
for page in range(2, 3):
yield scrapy.Request("http://www.ebay.com/sch/Cars-Trucks-/6001/i.html?_pgn=%d&_skc=50&rt=nc" % page,
callback=self.parse,
dont_filter=True)
def parse(self, response):
for car in response.xpath('//a[@class="vip"]'):
item = AmazonItem()
item['Name'] = car.xpath("@title").extract()[0]
item['Source'] = car.xpath("@href").extract()[0]
yield item
将其运行为 scrapy runningpider myspider.py -o output.json -t json 后，我在 output.json 中获得了以下内容(100 个唯一链接) ):
[{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/262100786610?hash=item3d066c9db2:g:9jUAAOSw14xWJAxz&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-Convertible-2-Door-/262107754535?hash=item3d06d6f027:g:kbEAAOSw~bFWJaB8&vxp=mtr", "Name": "Click this link to access Ford : Mustang Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/111804349250?hash=item1a080efb42:g:jZIAAOSw14xWKZa0&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/252137415321?hash=item3ab48fa299:g:nWkAAOSwo6lWKXGi&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-base-coupe-2-door-/301765507926?hash=item4642a03b56:g:PhYAAOSwFnFWA25u&vxp=mtr", "Name": "Click this link to access Ford : Mustang base coupe 2-door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-5-Series-Base-Sedan-4-Door-/121794866401?hash=item1c5b8a2ce1:g:0fEAAOSw5VFWKVGP&vxp=mtr", "Name": "Click this link to access BMW : 5-Series Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-GT-2dr-Convertible-/231726884262?hash=item35f3ffa5a6:g:1bgAAOSwI-BWJsVR&vxp=mtr", "Name": "Click this link to access Ford : Mustang GT 2dr Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-GT-Coupe-2-Door-/331687140950?hash=item4d3a181e56:g:EP8AAOSwA4dWJvaZ&vxp=mtr", "Name": "Click this link to access Ford : Mustang GT Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-2dr-Coupe-/151836461733?hash=item235a288aa5:g:eTkAAOSw9mFWJ-RP&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-i8-Base-Coupe-2-Door-/252120259686?hash=item3ab389dc66:g:9hIAAOSwT5tWGXvQ&vxp=mtr", "Name": "Click this link to access BMW : i8 Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Rolls-Royce-Other-Base-Coupe-2-Door-/252120246685?hash=item3ab389a99d:g:FAcAAOSwu4BV5fiN&vxp=mtr", "Name": "Click this link to access Rolls-Royce : Other Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Stingray-Coupe-2-Door-/272021279196?hash=item3f55bb51dc:g:Uf0AAOSwAYtWJVWJ&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Stingray Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-3-Series-Base-Sedan-4-Door-/141807178473?hash=item21045dd2e9:g:5z0AAOSw9mFWJuG1&vxp=mtr", "Name": "Click this link to access BMW : 3-Series Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-COUPE-TWO-DOOR-/221918999034?hash=item33ab6731fa:g:xPUAAOSwhcJWJaNt&vxp=mtr", "Name": "Click this link to access Ford : Other COUPE TWO DOOR"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/111800146014?hash=item1a07ced85e:g:DzIAAOSwT5tWIrOZ&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Dodge-Other-4-door-sedan-/121793219071?hash=item1c5b7109ff:g:NGkAAOSwAYtWJtSd&vxp=mtr", "Name": "Click this link to access Dodge : Other 4 door sedan"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/121794609975?hash=item1c5b864337:g:GY8AAOSw14xWKQoh&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2DR-COUPE-/252111788067?hash=item3ab3089823:g:akoAAOSwNgxWD06F&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2DR COUPE"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-S-10-Base-2-door-/281835962068?hash=item419ebb7ed4:g:3~EAAOSwYHxWH-BV&vxp=mtr", "Name": "Click this link to access Chevrolet : S-10 Base 2 door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Other-Makes-2dr-Coupe-/351538461712?hash=item51d9533c10:g:wC4AAOSw9r1WEDeu&vxp=mtr", "Name": "Click this link to access Other Makes 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-2DR-CONVERTIBLE-/252111729868?hash=item3ab307b4cc:g:j1kAAOSwA4dWG-Ns&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro 2DR CONVERTIBLE"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Coupe-C4-/401014005654?hash=item5d5e4c1f96:g:~lIAAOSwI-BWJldW&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Coupe C4"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Fusion-SE-Sedan-4-Door-/181908854518?hash=item2a5a9cd6f6:g:P6MAAOSwo6lWJvMQ&vxp=mtr", "Name": "Click this link to access Ford : Fusion SE Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/191719555199?hash=item2ca360407f:g:zysAAOSw5ZBWF~~K&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-3-Series-Base-Sedan-4-Door-/252133172015?hash=item3ab44ee32f:g:pYcAAOSw~bFWJZTK&vxp=mtr", "Name": "Click this link to access BMW : 3-Series Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/321895583306?hash=item4af278ce4a:g:vrsAAOSwAYtWIYzl&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Coupe-2-Door-/221916135811?hash=item33ab3b8183:g:M-sAAOSwhcJWIl~Z&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-GT-Coupe-2-Door-/231728815516?hash=item35f41d1d9c:g:GcEAAOSw4HVWEy51&vxp=mtr", "Name": "Click this link to access Ford : Mustang GT Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/262104898215?hash=item3d06ab5aa7:g:50YAAOSwsB9WEaZH&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/161866469654?hash=item25affe5116:g:RO8AAOSwI-BWKUFD&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-7-Series-Base-Sedan-4-Door-/111801265276?hash=item1a07dfec7c:g:fPcAAOSwHnFV2qVp&vxp=mtr", "Name": "Click this link to access BMW : 7-Series Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Audi-Other-S-Sedan-2-Door-/221915837097?hash=item33ab36f2a9:g:lsYAAOSw9mFWIc3o&vxp=mtr", "Name": "Click this link to access Audi : Other S Sedan 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Stingray-Coupe-2-Door-/272021258361?hash=item3f55bb0079:g:KeMAAOSw~bFWJU4K&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Stingray Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2DR-COUPE-/252114117664?hash=item3ab32c2420:g:aK0AAOSwdvpWEojh&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2DR COUPE"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-F-250-4WD-SuperCab-/131634437382?hash=item1ea6061d06:g:FxYAAOSwyQtVz5Ku&vxp=mtr", "Name": "Click this link to access Ford : F-250 4WD SuperCab"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-2-door-/161862555533?hash=item25afc2978d:g:D8UAAOSwo0JWJD7u&vxp=mtr", "Name": "Click this link to access Ford : Other 2 door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Impala-2-Door-Coupe-/171975493473?hash=item280a89c761:g:4GIAAOSwT6pVz1xF&vxp=mtr", "Name": "Click this link to access Chevrolet : Impala 2 Door Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-Pickups-F1-/272021449898?hash=item3f55bdecaa:g:MgAAAOSwo6lWJZ5A&vxp=mtr", "Name": "Click this link to access Ford : Other Pickups F1"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Convertible-/351552933325?hash=item51da300dcd:g:r-EAAOSw5VFWInPR&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/Dodge-Challenger-2DR-COUPE-/262106149321?hash=item3d06be71c9:g:AdsAAOSwT5tWKFT4&vxp=mtr", "Name": "Click this link to access Dodge : Challenger 2DR COUPE"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Coupe-w-1LT-/262092854617?hash=item3d05f39559:g:tXIAAOSw~bFWHbgc&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Coupe w/1LT"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Convertible-/301776873624?hash=item46434da898:g:QyIAAOSwhcJWKaL8&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2DR-COUPE-/252111744678?hash=item3ab307eea6:g:SUAAAOSwFnFWDzvG&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2DR COUPE"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-M5-Base-Sedan-4-Door-/301766757925?hash=item4642b34e25:g:-hwAAOSwwbdWHHVE&vxp=mtr", "Name": "Click this link to access BMW : M5 Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Nova-Base-Coupe-2-Door-/201451474018?hash=item2ee7718c62:g:BGwAAOSw9mFWJBkd&vxp=mtr", "Name": "Click this link to access Chevrolet : Nova Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-LX-Notch-Notchback-Coupe-/231726128599?hash=item35f3f41dd7:g:PkoAAOSwwbdWJa36&vxp=mtr", "Name": "Click this link to access Ford : Mustang LX Notch Notchback Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-Z-28-Coupe-2-Door-/231721774564?hash=item35f3b1ade4:g:cdYAAOSwAYtWIAUv&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro Z/28 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Audi-Other-Base-Coupe-2-Door-/151855137581?hash=item235b45832d:g:1C4AAOSwo0JWJozC&vxp=mtr", "Name": "Click this link to access Audi : Other Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Coupe-2-door-/121791512235?hash=item1c5b56feab:g:gAMAAOSwo0JWJDY2&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Coupe 2-door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-Pickups-4dr-Crew-Cab-Pickup-/151857070181?hash=item235b630065:g:c7AAAOSwl9BWKFdI&vxp=mtr", "Name": "Click this link to access Ford : Other Pickups 4dr Crew Cab Pickup"},
{"Source": "http://cgi.ebay.com/ebaymotors/Cadillac-XLR-Base-Convertible-2-Door-/252134088622?hash=item3ab45cdfae:g:iyQAAOSwHjNWArFm&vxp=mtr", "Name": "Click this link to access Cadillac : XLR Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Cadillac-STS-Base-Sedan-4-Door-/111801871064?hash=item1a07e92ad8:g:aC4AAOSwKsRWFxDN&vxp=mtr", "Name": "Click this link to access Cadillac : STS Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Audi-Other-Base-Coupe-2-Door-/151855137581?hash=item235b45832d:g:1C4AAOSwo0JWJozC&vxp=mtr", "Name": "Click this link to access Audi : Other Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Lamborghini-Gallardo-Base-Coupe-2-Door-/262104215977?hash=item3d06a0f1a9:g:snAAAOSwEetWA17N&vxp=mtr", "Name": "Click this link to access Lamborghini : Gallardo Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-2dr-Coupe-/181905760097?hash=item2a5a6d9f61:g:XkEAAOSwwbdWInNX&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Falcon-Sedan-4-door-/301776527754?hash=item464348618a:g:YKkAAOSwuTxV-i-o&vxp=mtr", "Name": "Click this link to access Ford : Falcon Sedan, 4 door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-Pickups-4dr-Crew-Cab-Pickup-/151857070181?hash=item235b630065:g:c7AAAOSwl9BWKFdI&vxp=mtr", "Name": "Click this link to access Ford : Other Pickups 4dr Crew Cab Pickup"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/321883289829?hash=item4af1bd38e5:g:Wd8AAOSwHmhV8yh~&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Bentley-Mulsanne-Base-Sedan-4-Door-/262090808466?hash=item3d05d45c92:g:1DoAAOSwAYtWHB68&vxp=mtr", "Name": "Click this link to access Bentley : Mulsanne Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-2dr-Coupe-/351537325474?hash=item51d941e5a2:g:h7cAAOSwKIpWDvcM&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-SS-Base-Sedan-4-Door-/151855140276?hash=item235b458db4:g:Ob0AAOSw~bFWJo18&vxp=mtr", "Name": "Click this link to access Chevrolet : SS Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-Z28-Coupe-2-Door-/221917387498?hash=item33ab4e9aea:g:34EAAOSwQPlV8kMg&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro Z28 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-7-Series-Base-Sedan-4-Door-/262103922812?hash=item3d069c787c:g:4rcAAOSwKIpV~hRw&vxp=mtr", "Name": "Click this link to access BMW : 7-Series Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Honda-Civic-EX-Sedan-4-Door-/252132849165?hash=item3ab449f60d:g:CD0AAOSwo0JWF~cE&vxp=mtr", "Name": "Click this link to access Honda : Civic EX Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Convertible-/351552935629?hash=item51da3016cd:g:Jk8AAOSw9mFWInUM&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-M5-Base-Sedan-4-Door-/301773611092?hash=item46431be054:g:MXUAAOSw5VFWJpd~&vxp=mtr", "Name": "Click this link to access BMW : M5 Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Convertible-/351552936138?hash=item51da3018ca:g:iM0AAOSw~bFWInVP&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/Nissan-350Z-Touring-Coupe-2-Door-/201452410603?hash=item2ee77fd6eb:g:vmwAAOSwhcJWJkkX&vxp=mtr", "Name": "Click this link to access Nissan : 350Z Touring Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Honda-S2000-2DR-Convertible-/111801087388?hash=item1a07dd359c:g:WgoAAOSw8d9UwwfV&vxp=mtr", "Name": "Click this link to access Honda : S2000 2DR Convertible"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-SS-Coupe-2-Door-/391299635374?hash=item5b1b4698ae:g:uYAAAOSwo0JWJufd&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro SS Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-GT-Coupe-2-Door-/281833278382?hash=item419e928bae:g:XdkAAOSwVL1WEqU7&vxp=mtr", "Name": "Click this link to access Ford : Mustang GT Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-COUPE-STAGE-3-/391298814909?hash=item5b1b3a13bd:g:elgAAOSwl9BWJbNp&vxp=mtr", "Name": "Click this link to access Ford : Mustang COUPE STAGE 3"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-Base-Coupe-2-Door-/201449891758?hash=item2ee75967ae:g:DtMAAOSw~bFWIIu5&vxp=mtr", "Name": "Click this link to access Ford : Mustang Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Other-coupe-/321899495746?hash=item4af2b48142:g:exsAAOSw9mFWJ9cv&vxp=mtr", "Name": "Click this link to access Ford : Other coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Other-Pickups-pickup-/281834256971?hash=item419ea17a4b:g:m9QAAOSwT5tWJsP2&vxp=mtr", "Name": "Click this link to access Chevrolet : Other Pickups pickup"},
{"Source": "http://cgi.ebay.com/ebaymotors/Honda-Civic-EX-L-Sedan-4-Door-/171972691648?hash=item280a5f06c0:g:rYMAAOSw9mFWJBWq&vxp=mtr", "Name": "Click this link to access Honda : Civic EX-L Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/321887782110?hash=item4af201c4de:g:qZ4AAOSwyQtVlUZU&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Toyota-Camry-LE-Sedan-4-Door-/281831345200?hash=item419e750c30:g:wo8AAOSwo0JWIslF&vxp=mtr", "Name": "Click this link to access Toyota : Camry LE Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Z51-Coupe-2-Door-/331685685604?hash=item4d3a01e964:g:P-kAAOSwKsRWFpFv&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Z51 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Audi-S5-Base-Coupe-2-Door-/301773650019?hash=item46431c7863:g:BuUAAOSwT5tWJqH1&vxp=mtr", "Name": "Click this link to access Audi : S5 Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-ZR1-Coupe-2-Door-/272021298479?hash=item3f55bb9d2f:g:6BEAAOSw37tWEriY&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette ZR1 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Porsche-914-Base-Coupe-2-Door-/391298621813?hash=item5b1b372175:g:TbgAAOSwhcJWJVh3&vxp=mtr", "Name": "Click this link to access Porsche : 914 Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2dr-Coupe-w-3LT-/301768547978?hash=item4642ce9e8a:g:1OEAAOSwwbdWHxhG&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2dr Coupe w/3LT"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Coupe-2-Door-/272021286949?hash=item3f55bb7025:g:0sAAAOSwAYtWF~wy&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-2-Doors-Coupe-/321895348805?hash=item4af2753a45:g:4wAAAOSwT5tWISbQ&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette 2 Doors Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Bugatti-Veyron-2dr-Coupe-/181909727919?hash=item2a5aaa2aaf:g:I-cAAOSw~bFWKFLA&vxp=mtr", "Name": "Click this link to access Bugatti : Veyron 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Tahoe-2WD-4dr-/121789738283?hash=item1c5b3bed2b:g:iJsAAOSw3ydVy5-5&vxp=mtr", "Name": "Click this link to access Chevrolet : Tahoe 2WD 4dr"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Taurus-SE-Sedan-4-Door-/171975983304?hash=item280a9140c8:g:nT8AAOSwAYtWHUdm&vxp=mtr", "Name": "Click this link to access Ford : Taurus SE Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Camaro-Z28-Coupe-2-Door-/121792392174?hash=item1c5b646bee:g:FekAAOSw14xWGXQF&vxp=mtr", "Name": "Click this link to access Chevrolet : Camaro Z28 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Base-Convertible-2-Door-/321900418381?hash=item4af2c2954d:g:MVoAAOSwN81WES11&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Base Convertible 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Z51-Coupe-2-Door-/252130973086?hash=item3ab42d559e:g:5SsAAOSwKsRWFyMR&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Z51 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-M5-Base-Sedan-4-Door-/291584114878?hash=item43e3c480be:g:cT4AAOSw~bFWFdzx&vxp=mtr", "Name": "Click this link to access BMW : M5 Base Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Acura-NSX-Base-Coupe-2-Door-/321898095939?hash=item4af29f2543:g:lnoAAOSwI-BWJZL7&vxp=mtr", "Name": "Click this link to access Acura : NSX Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-F-150-4dr-Pickup-/181905771090?hash=item2a5a6dca52:g:PTQAAOSwA4dWIne8&vxp=mtr", "Name": "Click this link to access Ford : F-150 4dr Pickup"},
{"Source": "http://cgi.ebay.com/ebaymotors/Dodge-Charger-Police-Pursuit-Sedan-4-Door-/201451939813?hash=item2ee778a7e5:g:sswAAOSw5ZBWJTA5&vxp=mtr", "Name": "Click this link to access Dodge : Charger Police Pursuit Sedan 4-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chrysler-Crossfire-Base-Coupe-2-Door-/121782566636?hash=item1c5ace7eec:g:2-sAAOSwgQ9VrVBR&vxp=mtr", "Name": "Click this link to access Chrysler : Crossfire Base Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-F-450-4dr-Pickup-/151851933760?hash=item235b14a040:g:apwAAOSwo0JWInir&vxp=mtr", "Name": "Click this link to access Ford : F-450 4dr Pickup"},
{"Source": "http://cgi.ebay.com/ebaymotors/BMW-M4-2dr-Coupe-/351552927943?hash=item51da2ff8c7:g:YukAAOSw5ZBWJqsN&vxp=mtr", "Name": "Click this link to access BMW : M4 2dr Coupe"},
{"Source": "http://cgi.ebay.com/ebaymotors/Chevrolet-Corvette-Z06-Coupe-2-Door-/111801187527?hash=item1a07debcc7:g:q2cAAOSwl9BWJGEq&vxp=mtr", "Name": "Click this link to access Chevrolet : Corvette Z06 Coupe 2-Door"},
{"Source": "http://cgi.ebay.com/ebaymotors/Ford-Mustang-Mach-I-Coupe-2-Door-/111802805072?hash=item1a07f76b50:g:iLAAAOSwAYtWJtjA&vxp=mtr", "Name": "Click this link to access Ford : Mustang Mach I Coupe 2-Door"}]
关于python - 为什么 python scrapy 在我的代码中产生重复的项目？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/33293651/