twitter04

textmining

twitter

Published

October 28, 2022

Exercise

Laden Sie \(n=10^k\) Tweets von Twitter herunter (mit \(k=2\)) via der Twitter API; Suchterm soll sein “@karl_lauterbach”. Bereiten Sie die Textdaten mit grundlegenden Methoden des Textminings auf (Tokenisieren, Stopwörter entfernen, Zahlen entfernen, …). Berichten Sie dann die 10 häufigsten Wörter als Schätzer für die Dinge, die an Karl Lauterbach getweetet werden.

Solution

library(rtweet)
library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.3     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter()  masks stats::filter()
✖ purrr::flatten() masks rtweet::flatten()
✖ dplyr::lag()     masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

library(tidytext)
library(lsa)  # Stopwörter

Loading required package: SnowballC

library(SnowballC)  # Stemming

source("/Users/sebastiansaueruser/credentials/hate-speech-analysis-v01-twitter.R")

auth <- rtweet_app(bearer_token = Bearer_Token)

karl1 <- search_tweets("@karl_lauterbach", n = 1e2, include_rts = FALSE)
#write_rds(karl1, file = "karl1.rds", compress = "gz")

karl2 <- 
  karl1 %>% 
  select(full_text)

karl3 <- 
  karl2 %>% 
  unnest_tokens(output = word, input = full_text)

karl4 <- 
karl3 %>% 
  anti_join(tibble(word = lsa::stopwords_de))

Joining with `by = join_by(word)`

karl5 <- 
  karl4 %>% 
  mutate(word = str_replace_na(word, "^[:digit:]+$")) %>% 
  mutate(word = str_replace_na(word, "hptts?://\\w+")) %>% 
  mutate(word = str_replace_na(word, " +")) %>% 
  drop_na()

karl6 <-
  karl5 %>% 
  mutate(word = wordStem(word))

karl6 %>% 
  count(word, sort = TRUE) %>% 
  slice_head(n=10)

# A tibble: 10 × 2
   word                       n
   <chr>                  <int>
 1 karl_lauterbach          100
 2 rt                        60
 3 ultrakaerl                19
 4 corona                    16
 5 wirwollenmaskenpflicht    16
 6 länder                    12
 7 gesundheitsminist         11
 8 polarstern64              11
 9 schon                     11
10 shomburg                  11

Categories:

textmining
twitter