Extract Domains using regex

Q&A
,
#1

Screen Link:
https://app.dataquest.io/m/369/advanced-regular-expressions/8/extracting-domains-from-urls

My Code: 

test_urls = pd.Series([
 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429',
 'http://www.interactivedynamicvideo.com/',
 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0',
 'http://evonomics.com/advertising-cannot-maintain-internet-heres-solution/',
 'HTTPS://github.com/keppel/pinn',
 'Http://phys.org/news/2015-09-scale-solar-youve.html',
 'https://iot.seeed.cc',
 'http://www.bfilipek.com/2016/04/custom-deleters-for-c-smart-pointers.html',
 'http://beta.crowdfireapp.com/?beta=agnipath',
 'https://www.valid.ly?param',
 'http://css-cursor.techstream.org'
])
pattern = r"//(\w+\-?\w+?\.?\w+\.?\w+\.?\w+)" 
test_urls_clean = test_urls.str.extract(pattern, flags=re.I)
test_urls_clean

domains = hn['url'].str.extract(pattern,flags=re.I)
top_domains = domains.value_counts()
top_domains.head(5)

What I expected to happen:
github.com 1008
medium.com 825
www.nytimes.com 525
www.theguardian.com 248
techcrunch.com 245

What actually happened:

top_domains is longer than we expected.

I m not sure what is going wrong, the regex pattern r"//(\w+\-?\w+?\.?\w+\.?\w+\.?\w+)" which I have created gives same output than regex pattern r"https?://([\w\-\.]+)" given in answer, than also in console it is stating top domains is longer than we expected, also it shows error for domain variable,
Please help me to understand difference between both the regex pattern
have attached snap for reference-
dq

1 Like
#2

Here’s a hint.

Take a look at the index 20011 in hn — the URL is https://www.linux-toys.com/?p=374.

What do you think the solution should capture? What does your solution capture?

1 Like
#3

Hi Bruno,
Thank you for looking into it, the url shared for reference is not available, though i will again check my regex pattern as per your guidance

1 Like
#4

What do you mean it’s not available? I see it:

image

If it isn’t there for you, there’s something else going on.

1 Like
#5

Thank you for coming back,
I thought you are referring me a url to browse, so I said it is not available, as I was not able to browse
Now I got it, you were referring me to url in dataset at given location