In the last decade, several methods have tackled the challenge of reconstructing gene regulatory networks from gene expression data. Several papers have compared and evaluated the different network inference methods relying on simulated data.

This is a new comparison that assesses different methods in a high-heterogeneity data scenario which could reveal the specialization of methods for the different network types and data.

This package allows repeating the comparison between different network inference algorithms with only one line of code.

This package allows replication this comparison between the different networks inference algorithms with only one line of code.

Toy example for main benchmark:

   
library(netbenchmark)
top20.aupr <- netbenchmark(methods="all",datasources.names = "Toy",
                                                     local.noise=20,global.noise=10,
                                                     noiseType=c("normal","lognormal"),
                                                     datasets.num = 2,experiments = 40,
                                                     seed=1422976420,verbose=FALSE)

The first element of the returned list is the Area Under the Precison Recall (AUPR) on the 
20% most confident predictions (AUPR20) :

print(top20.aupr[[1]])
##   Origin experiments  aracne c3net   clr    GeneNet
## 1    toy       48     0.1486 0.1308 0.1714   0.0558
## 2    toy       35     0.0928 0.0935 0.1172   0.0523
##     Genie3  mrnet   mutrank   mrnetb pcit    zscore
## 1   0.1786  0.1702   0.1190   0.1760 0.1697  0.03308
## 2   0.1425  0.1197   0.0748   0.1217 0.1419  0.0147
##     rand
## 1   0.0220
## 2   0.0232

The package provides an easy way to compare new techniques with state-of-the-art ones 
and to make new different benchmarks in the future. First, define the wrapper functions:

Spearmancor <- function(data){
                  cor(data,method="spearman")
}

Pearsoncor <- function(data){
        cor(data,method="pearson")
}

Note that the wrapper function returns a matrix which is the weighted adjacency matrix of 
the network inferred by the algorithm and that the columns and rows are named. 
Evaluate five times these two simple inference methods with syntren300 datasource:

res <- netbenchmark(datasources.names="syntren300",
                                        methods=c("Spearmancor","Pearsoncor"),verbose=FALSE)
aupr <- res[[1]][,-(1:2)]

Make a boxplot of the AUPR20 results:

 boxplot(aupr, main="Syntren300",ylab=expression('AUPR'[20]))

Plot the mean Precision-Recall curves:

We can also compare these two simple inference methods with the fast network inference algorithms using syntren300 datasource:

aupr <- comp[[1]][,-(1:2)]

Make a boxplot the AUPR20 results:

 #make the name look prety
library("tools")
colnames(aupr) <- sapply(colnames(aupr),file_path_sans_ext)
boxplot(aupr, main="Syntren300", ylab=expression('AUPR'[20])