※ 引述《yuseke (yuseke)》之铭言：
: as title
: 这两天在看RE的部分
: 根据PYTHON 基金会的网站:
: https://docs.python.org/2/library/re.html
: 7.2.5.7. Finding all Adverbs and their Positions¶
: If one wants more information about all matches of a pattern than the matched
: text, finditer() is useful as it provides instances of MatchObject instead of
: strings. Continuing with the previous example, if one was a writer who wanted
: to find all of the adverbs and their positions in some text, he or she would
: use finditer() in the following manner:
: ... print '%02d-%02d: %s' % (m.start(), m.end(), m.group(0))
: 07-16: carefully
: 40-47: quickly
: 关于""""for m in re.finditer(r"\w+ly", text):""""
: 这个部份我有一个疑问,
: 可是并不是所有的adv都有ly字尾.......
: 这种情形该怎么处理呢?
在自然语言处理中，一个常用的方法是建机率模型
就是统计连续二个或三个POS出现的组合
假设训练资料中，(AT, NN)后面出现的机率最高是IN
则遇到(AT,NN,??)的时候，就填NN
一般会用trigram tagger，以三个POS为主
如果test data出现没见过的POS组合，就查bigram tagger
如果再查不到，就一律视为NN
下面是用brown corpus作训练资料，作一个简单的tagger
也可以参考下面的网页
http://www.nltk.org/book/ch05.html
import nltk
train = nltk.corpus.brown.tagged_sents()
train[0]
# [(u'The', u'AT'),
# (u'Fulton', u'NP-TL'),
# (u'County', u'NN-TL'),
# (u'Grand', u'JJ-TL'),
# (u'Jury', u'NN-TL'),
# (u'said', u'VBD'),
# (u'Friday', u'NR'),
# (u'an', u'AT'),
# (u'investigation', u'NN'),
# (u'of', u'IN'),
# (u"Atlanta's", u'NP$'),
# (u'recent', u'JJ'),
# (u'primary', u'NN'),
# (u'election', u'NN'),
# (u'produced', u'VBD'),
# (u'``', u'``'),
# (u'no', u'AT'),
# (u'evidence', u'NN'),
# (u"''", u"''"),
# (u'that', u'CS'),
# (u'any', u'DTI'),
# (u'irregularities', u'NNS'),
# (u'took', u'VBD'),
# (u'place', u'NN'),
# (u'.', u'.')]
# backoff是遇到没遇过的组合时，要作的动作
default_tagger = nltk.DefaultTagger('NN')
bi_tagger = nltk.BigramTagger(train, backoff=default_tagger)
tri_tagger = nltk.TrigramTagger(train, backoff=bi_tagger)
# 用训练出的tagger测试
tri_tagger.tag('John is a dog'.split())
# [('John', u'NP'), ('is', u'BEZ'), ('a', u'AT'), ('dog', 'NN')]