The R code below can be used to clean your text : # Convert the text to lower caseĭocs <- tm_map(docs, content_transformer(tolower))ĭocs <- tm_map(docs, removeWords, stopwords("english")) Note that, text stemming require the package ‘SnowballC’. For example, a stemming process reduces the words “moving”, “moved” and “movement” to the root word, “move”. In other words, this process removes suffixes from words to make it simple and to get the common origin.
You could also remove numbers and punctuation with removeNumbers and removePunctuation arguments.Īnother important preprocessing step is to make a text stemming which reduces words to their root form. I’ll also show you how to make your own list of stopwords to remove from the text. For ‘stopwords’, supported languages are danish, dutch, english, finnish, french, german, hungarian, italian, norwegian, portuguese, russian, spanish and swedish. Removing this kind of words is useful before further analyses. The information value of ‘stopwords’ is near zero due to the fact that they are so common in a language.
The tm_map() function is used to remove unnecessary white space, to convert the text to lower case, to remove common stopwords like ‘the’, “we”.
0 kommentar(er)
