GraphFramEx
A systematic evaluation framework for explainability methods on Graph Neural Networks
The goal of GraphFramEx is to systematically evaluate
methods that generate explanations for predictions
of graph neural networks (GNNs).
GraphFramEx proposes a unique metric, the characterization score, which combines the
fidelity measures, and classifies explanations based on their quality of being sufficient
or necessary. We scope ourselves to node classification tasks and compare the most
representative techniques in the field of input-level explainability for GNNs.
Why is it important to have a rigorous comparison of explainability methods?
The past decade has witnessed the rise of many
new methods
to explain GNN predictions
, but it is still unclear how to compare them and how to evalute their quality.
Additionally,
we open source the
GraphFramEx library
that compares diverse explainability methods used for the leaderboard to facilitate their
usage for downstream applications.
To add methods to this comparison and enrich our knowledge on explainability,
we also welcome external methods. To add your method, please go to the
Up-to-date leaderboard based
on 10+ explainability methods
Integrate your new explainability method
and learn
how competitive your method is!
Leaderboard: Cora, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Citeseer, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Pubmed, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Cornell, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Texas, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Wisconsin, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Facebook, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Actor, focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Squirrel (Wikipedia), focus: Phenomenon, mask: Hard, size: 10 edges
Leaderboard: Chameleon (Wikipedia), focus: Phenomenon, mask: Hard, size: 10 edges
Citation
@article{amara2022graphframex, title={GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks}, author={Amara, Kenza and Ying, Rex and Zhang, Zitao and Han, Zhihao and Shan, Yinan and Brandes, Ulrik and Schemm, Sebastian and Zhang, Ce}, journal={arXiv preprint arXiv:2206.09677}, year={2022} }
Add your method to GraphFramEx!
