Skip to main content
 首页 » 编程设计

python之从不同结构和工资符号的 HTML 工作规范中提取工资

2025年05月04日10mfrbuaa

我希望尽可能以通用的方式从工作规范中提取薪水信息(考虑到可以指定薪水的多种方式(前面有或没有“薪水”一词,我们没有尾随零、范围等)

采用三种不同的工作规范,我用 urllib2 获取 HTML , 然后执行一个不区分大小写的初始 grep对于“每个人的薪水”一词。结果差异很大(请原谅 Jupyter 的不太漂亮的粘贴):

In [52]: 
 
urllib2 
Out[52]: 
<module 'urllib2' from '/Users/Evan/anaconda/lib/python2.7/urllib2.pyc'> 
In [82]: 

案例#1

reponse = urllib2_urlopen('http://apply.ovoenergycareers.co.uk/vacancies/453/cro-manager/london/') 
In [83]: 
 
content = reponse.read() 
In [84]: 
 
save_html('salarygrep1', content) 
In [59]: 
 
!grep -i salary salarygrep1.html 
!grep -i salary salarygrep1.html 
  <dt class="field_salary">Salary</dt> 
  <dd class="value_salary"> 
In [86]: 
 
with open('salarygrep1.html') as s: 
    for line in s: 
        if 'salary' in line.lower(): 
            print line 
  <dt class="field_salary">Salary</dt> 
 
  <dd class="value_salary"> 
 
In [79]: 

案例 #2

reponse = urllib2_urlopen('http://apply.ovoenergycareers.co.uk/vacancies/475/ovo-telesales-agent/bristol/') 
In [80]: 
 
content = reponse.read() 
In [81]: 
 
save_html('salarygrep2', content) 
In [63]: 
 
!grep -i salary salarygrep2.html 
  <dt class="field_salary">Salary</dt> 
  <dd class="value_salary"> 
    Salary: �18,000 + benefits & competitive commission scheme; OTE range: �20,500 - �30,000 
In [87]: 
 
with open('salarygrep2.html') as s: 
    for line in s: 
        if 'salary' in line.lower(): 
            print line 
​ 
  <dt class="field_salary">Salary</dt> 
 
  <dd class="value_salary"> 
 
    Salary: �18,000 + benefits & competitive commission scheme; OTE range: �20,500 - �30,000 
 
In [88]: 

案例 #3

reponse = urllib2_urlopen('https://gs7.globalsuccessor.com/centrica02/tpl_centrica02.asp?s=4A515F4E5A565B1A&jobid=48490,2356610248&key=21798303&c=028859657862&pagestamp=dbykvxmmwfnblykbqc') 
In [89]: 
 
content = reponse.read() 
In [90]: 
 
save_html('salarygrep3', content) 
In [67]: 
 
!grep -i salary salarygrep3.html 
!grep -i salary salarygrep3.html 
<p id="igSoundBite"><em><div>Salary: &#163;28-&#163;38K depending on experience</div></em></p><h3 id="igJobDesc0">Job Description</h3><p><div>Assistant Product Development Manager </div> 
In [95]: 
 
with open('salarygrep3.html') as s: 
    for line in s: 
        if 'salary' in line.lower(): 
            print line 
<p id="igSoundBite"><em><div>Salary: &#163;28-&#163;38K depending on experience</div></em></p><h3 id="igJobDesc0">Job Description</h3><p><div>Assistant Product Development Manager </div> 
 
In [70]: 

案例 #4

reponse = urllib2_urlopen('http://jobs.emounlimited.com/senior-digital-project-manager/') 
In [71]: 
 
content = reponse.read() 
In [72]: 
 
save_html('salarygrep4', content) 
In [94]: 
 
!grep -i salary salarygrep4.html 
In [92]: 
 
with open('salarygrep4.html') as s: 
    for line in s: 
        if 'salary' in line.lower(): 
            print line 
In [ ]: 

  • 在案例 #1 中,实际工资在不同的 <div> 中, 而不是 被捡起来。
  • 在情况 #2 中,确实检测到了薪水,但 (i) 这是一个需要处理的范围,并且 (ii) 存在非 ASCII 货币字符(英国英镑)。
  • 在案例 #3 中,薪水也被检测到,但 (i) 它是一个需要处理的范围,并且 (ii) 存在非 ASCII 货币字符(英国英镑)。
  • 以防万一,因为指定了薪水但没有提到“薪水”这个词,所以没有检测到任何东西。

考虑到页面设计和薪水规范的广泛差异,认为通用(或通用)正则表达式或正则表达式组合可能会成功,这是否不合理?如果没有,我将如何构建它/它们?或者实际上,是否有一种 Python 方法可以减少对正则表达式的依赖?

请您参考如下方法:

这是一个想法:

  • 通过 BeautifulSoup 传递页面的 HTML
  • 获取 body 元素的文本(我们对其余部分不感兴趣)
  • 使用正则表达式,选择金额

代码:

# -*- coding: utf-8 -*- 
import re 
 
import requests 
from bs4 import BeautifulSoup 
 
urls = [ 
    "http://apply.ovoenergycareers.co.uk/vacancies/453/cro-manager/london/", 
    "http://apply.ovoenergycareers.co.uk/vacancies/475/ovo-telesales-agent/bristol/", 
    "https://gs7.globalsuccessor.com/centrica02/tpl_centrica02.asp?s=4A515F4E5A565B1A&jobid=48490,2356610248&key=21798303&c=028859657862&pagestamp=dbykvxmmwfnblykbqc", 
    "http://jobs.emounlimited.com/senior-digital-project-manager/" 
] 
 
money_pattern = re.compile(ur"($|£)([0-9.,]+K?)(?:\s*-\s*(?:$|£)*([0-9.,]+K?)*)*") 
for url in urls: 
    soup = BeautifulSoup(requests.get(url).text, "html.parser") 
    text = soup.body.text 
 
    print("URL: " + url) 
    for currency, amount1, amount2 in money_pattern.findall(text): 
        if not amount1 and not amount2: 
            continue 
 
        if not amount2: 
            print("Single amount found: %s, currency: %s" % (amount1, currency)) 
        else: 
            print("Range found: %s - %s, currency: %s" % (amount1, amount2, currency) ) 
    print("------") 

输出:

URL: http://apply.ovoenergycareers.co.uk/vacancies/453/cro-manager/london/ 
Range found: 40,000 - 50,000, currency: £ 
------ 
URL: http://apply.ovoenergycareers.co.uk/vacancies/475/ovo-telesales-agent/bristol/ 
Single amount found: 18,000, currency: £ 
Range found: 20,500 - 30,000, currency: £ 
------ 
URL: https://gs7.globalsuccessor.com/centrica02/tpl_centrica02.asp?s=4A515F4E5A565B1A&jobid=48490,2356610248&key=21798303&c=028859657862&pagestamp=dbykvxmmwfnblykbqc 
Range found: 28 - 38K, currency: £ 
------ 
URL: http://jobs.emounlimited.com/senior-digital-project-manager/ 
Range found: 36 - 40,000, currency: £ 
------ 

希望这至少能让您入门。