Biterm most frequent topic filtering

[BenoitFayolle]

textplot/R/textplot_biterms.R

Lines 230 to 231 in d0c40fb

	biterms <- biterms[, topic_freq := .N, by = list(term1, term2)]
	biterms <- biterms[, list(best_topic = topic[which.max(topic_freq)], cooc = .N), by = list(term1, term2)]

Correct me if I'm wrong but these don't actually pick the best/most frequent topic.
topic_freq gives the number of occurrences of each biterm in the whole corpus since topic is not included in the by argument of the first line.
Hence second line picks the maximum of a variable that is constant within each group

[jwijffels]

I tried to speed up the logic which I orignally implemented at

textplot/R/textplot_biterms.R

Lines 232 to 233 in d0c40fb

	# biterms <- biterms[, list(best_topic = utils::head(base::names(base::sort(base::table(topic), decreasing = TRUE)), 1),
	# cooc = .N), by = list(term1, term2)]

but the implementation wasn't the correct speedup.
Later on, I make sure only biterms with terms highly emitted by each topic are shown at

textplot/R/textplot_biterms.R

Line 246 in d0c40fb

bi <- bi[bi$term1 %in% topictokens & bi$term2 %in% topictokens, ]

This was done to make the graph crisp.
So a bug clearly but probably not occurring that much unless you really have completely overlapping topics.

[BenoitFayolle]

I think you are responding to my other issue but this one is different. I can send a reprex tomorrow

[BenoitFayolle]

Nevermind, I just saw your commit to fix this issue 👍

[jwijffels]

I pushed the package on CRAN just now.