* Determination of first isolates now **excludes** all 'unknown' microorganisms at default, i.e. microbial code `"UNKNOWN"`. They can be included with the new parameter `include_unknown`:
@ -72,6 +72,14 @@
@@ -72,6 +72,14 @@
```
### Changed
* Many algorithm improvements for `as.mo()` (of which some led to additions to the `microorganisms` data set):
* Self-learning algorithm - the function now gains experience from previously determined microorganism IDs and learns from it (yielding 80-95% speed improvement for any guess after the first try)
* Big improvement for misspelled input
* These new trivial names known to the field are now understood: meningococcus, gonococcus, pneumococcus
* Updated to the latest taxonomic data (updated to August 2019, from the International Journal of Systematic and Evolutionary Microbiology
* Added support for Viridans Group Streptococci (VGS) and Milleri Group Streptococci (MGS)
* Added support for 5,000 new fungi
* Added support for unknown yeasts and fungi
* Renamed data set `septic_patients` to `example_isolates`
* Function `eucast_rules()`:
* Fixed a bug for *Yersinia pseudotuberculosis*
@ -83,13 +91,6 @@
@@ -83,13 +91,6 @@
* Removed class `atc` - using `as.atc()` is now deprecated in favour of `ab_atc()` and this will return a character, not the `atc` class anymore
* Algorithm improvements for `as.mo()` (by which some additions were made to the `microorganisms` data set:
* Big improvement for misspelled input
* These new trivial names known to the field are now understood: meningococcus, gonococcus, pneumococcus
* Updated to the latest taxonomic data (updated to August 2019, from the International Journal of Systematic and Evolutionary Microbiology
* Added support for Viridans Group Streptococci (VGS) and Milleri Group Streptococci (MGS)
* Added support for 5,000 new fungi
* Added support for unknown yeasts and fungi
* Fix for using `mo_*` functions where the coercion uncertainties and failures would not be available through `mo_uncertainties()` and `mo_failures()` anymore
* Deprecated the `country` parameter of `mdro()` in favour of the already existing `guideline` parameter to support multiple guidelines within one country
* The `name` of `RIF` is now Rifampicin instead of Rifampin
#' Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive. Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors.
#' Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive (use \code{a \%like_case\% b} for case-sensitive matching). Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors, or can both have the same length to iterate over all cases.
#' @inheritParams base::grepl
#' @return A \code{logical} vector
#' @name like
@ -53,14 +53,14 @@
@@ -53,14 +53,14 @@
#' left_join_microorganisms() %>%
#' filter(genus %like% '^ent') %>%
#' freq(genus, species)
like<-function(x,pattern){
like<-function(x,pattern,ignore.case=TRUE){
if (length(pattern)>1){
if (length(x)!=length(pattern)){
if (length(x)==1){
x<-rep(x,length(pattern))
}
# return TRUE for every 'x' that matches any 'pattern', FALSE otherwise
@ -299,8 +301,11 @@ A microbial ID from this package (class: <code>mo</code>) typically looks like t
@@ -299,8 +301,11 @@ A microbial ID from this package (class: <code>mo</code>) typically looks like t
</pre>
<p>Values that cannot be coered will be considered 'unknown' and will get the MO code <code>UNKNOWN</code>.</p>
<p>Use the <code><ahref='mo_property.html'>mo_property</a>_*</code> functions to get properties based on the returned code, see Examples.</p>
<p>The algorithm uses data from the Catalogue of Life (see below) and from one other source (see <code><ahref='microorganisms.html'>?microorganisms</a></code>).</p>
<p><strong>Intelligent rules</strong><br/>
<p>The algorithm uses data from the Catalogue of Life (see below) and from one other source (see <code><ahref='microorganisms.html'>microorganisms</a></code>).</p>
<p><strong>Self-learning algoritm</strong><br/>
The <code>as.mo()</code> function gains experience from previously determined microorganism IDs and learns from it. This drastically improves both speed and reliability. Use <code>clear_mo_history()</code> to reset the algorithms. Only experience from your current <code>AMR</code> package version is used. This is done because in the future the taxonomic tree (which is included in this package) may change for any organism and it consequently has to rebuild its knowledge.</p>
<p>Usually, any guess after the first try runs 80-95% faster than the first try.</p>
<p><strong>Intelligent rules</strong><br/>
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:</p><ul>
<li><p>Valid MO codes and full names: it first searches in already valid MO code and known genus/species combinations</p></li>
<li><p>Human pathogenic prevalence: it first searches in more prevalent microorganisms, then less prevalent ones (see <em>Microbial prevalence of pathogens in humans</em> below)</p></li>
@ -326,7 +331,7 @@ The algorithm can additionally use three different levels of uncertainty to gues
@@ -326,7 +331,7 @@ The algorithm can additionally use three different levels of uncertainty to gues
</ul>
<p>Use <code>mo_failures()</code> to get a vector with all values that could not be coerced to a valid value.</p>
<p>Use <code>mo_uncertainties()</code> to get a data.frame with all values that were coerced to a valid value, but with uncertainty.</p>
<p>Use <code>mo_renamed()</code> to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name.</p>
<p>Use <code>mo_renamed()</code> to get a data.frame with all values that could be coerced based on an old, previously accepted taxonomic name.</p>
<p><strong>Microbial prevalence of pathogens in humans</strong><br/>
The intelligent rules take into account microbial prevalence of pathogens in humans. It uses three groups and all (sub)species are in only one group. These groups are:</p><ul>
<li><p>1 (most prevalent): class is Gammaproteobacteria <strong>or</strong> genus is one of: <em>Enterococcus</em>, <em>Staphylococcus</em>, <em>Streptococcus</em>.</p></li>
@ -334,7 +339,7 @@ The intelligent rules take into account microbial prevalence of pathogens in hum
@@ -334,7 +339,7 @@ The intelligent rules take into account microbial prevalence of pathogens in hum
<li><p>3 (least prevalent): all others.</p></li>
</ul>
<p>Group 1 contains all common Gram positives and Gram negatives, like all Enterobacteriaceae and e.g. <em>Pseudomonas</em> and <em>Legionella</em>.</p>
<p>Group 2 probably contains less microbial pathogens; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.</p>
<p>Group 2 contains probably less pathogenic microorganisms; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.</p>
<metaproperty="og:description"content="Convenient wrapper around grep to match a pattern: a %like% b. It always returns a logical vector and is always case-insensitive. Also, pattern (b) can be as long as x (a) to compare items of each index in both vectors."/>
<metaproperty="og:description"content="Convenient wrapper around grep to match a pattern: a %like% b. It always returns a logical vector and is always case-insensitive (use a %like_case% b for case-sensitive matching). Also, pattern (b) can be as long as x (a) to compare items of each index in both vectors, or can both have the same length to iterate over all cases."/>
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9067</span>
<spanclass="version label label-default"data-toggle="tooltip"data-placement="bottom"title="Latest development version">0.7.1.9073</span>
</span>
</div>
@ -230,13 +230,15 @@
@@ -230,13 +230,15 @@
<divclass="ref-description">
<p>Convenient wrapper around <code><ahref='https://www.rdocumentation.org/packages/base/topics/grep'>grep</a></code> to match a pattern: <code>a %like% b</code>. It always returns a <code>logical</code> vector and is always case-insensitive. Also, <code>pattern</code> (<code>b</code>) can be as long as <code>x</code> (<code>a</code>) to compare items of each index in both vectors.</p>
<p>Convenient wrapper around <code><ahref='https://www.rdocumentation.org/packages/base/topics/grep'>grep</a></code> to match a pattern: <code>a %like% b</code>. It always returns a <code>logical</code> vector and is always case-insensitive (use <code>a %like_case% b</code> for case-sensitive matching). Also, <code>pattern</code> (<code>b</code>) can be as long as <code>x</code> (<code>a</code>) to compare items of each index in both vectors, or can both have the same length to iterate over all cases.</p>
\item{x}{a character vector or a \code{data.frame} with one or two columns}
@ -45,7 +48,7 @@ Use this function to determine a valid microorganism ID (\code{mo}). Determinati
@@ -45,7 +48,7 @@ Use this function to determine a valid microorganism ID (\code{mo}). Determinati
}
\details{
\strong{General info} \cr
A microbial ID from this package (class: \code{mo}) typically looks like these examples:\cr
A microorganism ID from this package (class: \code{mo}) typically looks like these examples:\cr
@ -65,7 +68,13 @@ Values that cannot be coered will be considered 'unknown' and will get the MO co
@@ -65,7 +68,13 @@ Values that cannot be coered will be considered 'unknown' and will get the MO co
Use the \code{\link{mo_property}_*} functions to get properties based on the returned code, see Examples.
The algorithm uses data from the Catalogue of Life (see below) and from one other source (see \code{?microorganisms}).
The algorithm uses data from the Catalogue of Life (see below) and from one other source (see \code{\link{microorganisms}}).
\strong{Self-learning algoritm} \cr
The \code{as.mo()} function gains experience from previously determined microorganism IDs and learns from it. This drastically improves both speed and reliability. Use \code{clear_mo_history()} to reset the algorithms. Only experience from your current \code{AMR} package version is used. This is done because in the future the taxonomic tree (which is included in this package) may change for any organism and it consequently has to rebuild its knowledge.
Usually, any guess after the first try runs 80-95\% faster than the first try.
\strong{Intelligent rules} \cr
This function uses intelligent rules to help getting fast and logical results. It tries to find matches in this order:
@ -105,7 +114,7 @@ Use \code{mo_failures()} to get a vector with all values that could not be coerc
@@ -105,7 +114,7 @@ Use \code{mo_failures()} to get a vector with all values that could not be coerc
Use \code{mo_uncertainties()} to get a data.frame with all values that were coerced to a valid value, but with uncertainty.
Use \code{mo_renamed()} to get a vector with all values that could be coerced based on an old, previously accepted taxonomic name.
Use \code{mo_renamed()} to get a data.frame with all values that could be coerced based on an old, previously accepted taxonomic name.
\strong{Microbial prevalence of pathogens in humans} \cr
The intelligent rules take into account microbial prevalence of pathogens in humans. It uses three groups and all (sub)species are in only one group. These groups are:
@ -117,7 +126,7 @@ The intelligent rules take into account microbial prevalence of pathogens in hum
@@ -117,7 +126,7 @@ The intelligent rules take into account microbial prevalence of pathogens in hum
Group 1 contains all common Gram positives and Gram negatives, like all Enterobacteriaceae and e.g. \emph{Pseudomonas} and \emph{Legionella}.
Group 2 probably contains less microbial pathogens; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
Group 2 contains probably less pathogenic microorganisms; all other members of phyla that were found in humans in the Northern Netherlands between 2001 and 2018.
Idea from the \href{https://github.com/Rdatatable/data.table/blob/master/R/like.R}{\code{like} function from the \code{data.table} package}, but made it case insensitive at default and let it support multiple patterns. Also, if the regex fails the first time, it tries again with \code{perl = TRUE}.
}
\usage{
like(x, pattern)
like(x, pattern, ignore.case = TRUE)
x \%like\% pattern
x \%like_case\% pattern
}
\arguments{
\item{x}{a character vector where matches are sought, or an
@ -24,12 +27,15 @@ x \%like\% pattern
@@ -24,12 +27,15 @@ x \%like\% pattern
character vector of length 2 or more is supplied, the first element
is used with a warning. Missing values are allowed except for
\code{regexpr} and \code{gregexpr}.}
\item{ignore.case}{if \code{FALSE}, the pattern matching is \emph{case
sensitive} and if \code{TRUE}, case is ignored during matching.}
}
\value{
A \code{logical} vector
}
\description{
Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive. Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors.
Convenient wrapper around \code{\link[base]{grep}} to match a pattern: \code{a \%like\% b}. It always returns a \code{logical} vector and is always case-insensitive (use \code{a \%like_case\% b} for case-sensitive matching). Also, \code{pattern} (\code{b}) can be as long as \code{x} (\code{a}) to compare items of each index in both vectors, or can both have the same length to iterate over all cases.
}
\details{
Using RStudio? This function can also be inserted from the Addins menu and can have its own Keyboard Shortcut like Ctrl+Shift+L or Cmd+Shift+L (see Tools > Modify Keyboard Shortcuts...).
In the figure below, we compare *Escherichia coli* (which is very common) with *Prevotella brevis* (which is moderately common) and with *Thermus islandicus* (which is uncommon):
# In reality, the `as.mo()` functions **learns from its own output to speed up determinations for next times**. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use `as.mo()` yourself:
In reality, the `as.mo()` functions **learns from its own output to speed up determinations for next times**. In above figure, this effect was disabled to show the difference with the boxplot below - when you would use `as.mo()` yourself:
# The highest outliers are the first times. All next determinations were done in only thousands of seconds. For now, learning only works per session. If R is closed or terminated, the algorithms reset. This will probably be resolved in a next version.
```
The highest outliers are the first times. All next determinations were done in only thousands of seconds.
Uncommon microorganisms take a lot more time than common microorganisms. To relieve this pitfall and further improve performance, two important calculations take almost no time at all: **repetitive results** and **already precalculated results**.
