Traceback (most recent call last):
c_name = info_box.find('dt', text='Contact Person:').find_next_sibling('dd').text
AttributeError: 'NoneType' object has no attribute 'find_next_sibling'
What I have tried:
import
pandas
as
pd
import
requests
from
bs4
import
BeautifulSoup
headers = {
"
User-Agent"
:
"
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
big_list = []
for
i
in
range
(
1
,
50
):
url= f
"
https://www.usaopps.com/government_contractors/naics-111110-Soybean-Farming.{i}.htm"
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.text,
'
html.parser'
)
for
x
in
soup.select(
'
div.list-one'
)[:3]:
det_url =
'
https://www.usaopps.com'
+ x.select_one(
'
a'
).get(
'
href'
)
req = requests.get(det_url)
det_soup = BeautifulSoup(req.text,
'
html.parser'
)
info_box = det_soup.select_one(
'
div.info-gen-box'
)
c_name = info_box.find(
'
dt'
, text=
'
Contact Person:'
).find_next_sibling(
'
dd'
).text
c_fax = info_box.find(
'
dt'
, text=
'
Fax:'
).find_next_sibling(
'
dd'
).text
except
AttributeError
as
error:
print
(error)
else:
print
(c_name, c_fax)
big_list.append((c_name, c_fax))
df = pd.DataFrame(big_list, columns = [
'
Contact'
,
'
Fax'
])
When ever you get a problems that involves a message such as "
'nonetype' object has no attribute ..."
it means the same thing: you have tried to call a method on something that doesn't exist.
If you cann
find
and there is no such item, it returns a special value:
None
If you try to do anything with that value, you will get this error.
It's a bit like a pocket: you have a pocket in your shirt, which you use to hold a pen. If you reach into the pocket and find there isn't a pen there, you can't sign your name on a piece of paper - and you will get very funny looks if you try! The empty pocket is giving you a null value (no pen here!) so you can't do anything that you would normally do once you retrieved your pen. Why is it empty? That's the question - it may be that you forgot to pick up your pen when you left the house this morning, or possibly you left the pen in the pocket of yesterday's shirt when you took it off last night.
We can't tell, because we weren't there, and even more importantly, we can't even see your shirt, much less what is in the pocket!
Back to computers, and you have done the same thing, somehow - and we can't see your code, much less run it and find out what contains null when it shouldn't.
But you can - and the debugger will help you here. Run your program in the debugger and when it fails, it will show you the line it found the problem on. You can then start looking at the various parts of it to see what value is null and start looking back through your code to find out why. So put a breakpoint at the beginning of the method containing the error line, and run your program from the start again. This time, the debugger will stop before the error, and let you examine what is going on by stepping through the code looking at your values.
But we can't do that - we don't have your code, we don't know how to use it if we did have it, we don't have your data. So try it - and see how much information you can find out!
c_name = info_box.find(
'
dt'
, text=
'
Contact Person:'
).find_next_sibling(
'
dd'
).text
The message is telling you that
info_box.find
did not find anythings, so it returned
None
. And a
None
object does not have any properties or methods, so you cannot call
find_next_sibling
on it. When you use a method that may fail you should always check its return value before trying to use it in a pipeline.
Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad
spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or
edit the question
and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it.
Provide an answer or move on to the next question.
Let's work to help developers, not make them feel stupid.
