Visual Discrimination Puzzles Website #

CLEVR VDP Dataset #

The CLEVR dataset contains 825 visual discrimination puzzles and our solver’s performance on the same. The pipeline uses a MaskRCNN + Attribute network (from the NS-VQA project) as the backend neural module.

GQA VDP Dataset #

The GQA VDP dataset consists 5000 puzzles that were randomly generated using the GQA dataset. More information about the generation process can be found here.

Explore CLEVR Puzzles Explore GQA Puzzles

Image Licenses #

The GQA images are part of the COCO dataset. More information about the respective licenses for these images can be found here.

Citations #

The following external tools were used for this project:

  • GQA Dataset

        title={GQA: A New Dataset for Real-World Visual Reasoning 
        and Compositional Question Answering},
        author={Hudson, Drew A and Manning, Christopher D},
        journal={Conference on Computer Vision and Pattern Recognition (CVPR)},
  • NS-VQA inference pipeline

    Link: https://github.com/kexinyi/ns-vqa

    title={Neural-symbolic vqa: Disentangling reasoning from vision and language understanding},
    author={Yi, Kexin and Wu, Jiajun and Gan, Chuang and Torralba, Antonio and Kohli, Pushmeet and Tenenbaum, Joshua B.},
    booktitle={Advances in Neural Information Processing Systems},
  • CLEVR dataset generator

    Link: https://github.com/facebookresearch/clevr-dataset-gen

      title={CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning},
      author={Johnson, Justin and Hariharan, Bharath and van der Maaten, Laurens
              and Fei-Fei, Li and Zitnick, C Lawrence and Girshick, Ross},
  • Deep Ranking Similarity Baseline

    Link: https://github.com/USCDataScience/Image-Similarity-Deep-Ranking

    • Code
      Dr. Chris Mattmann, JPL.
      Purvak Lapsiya, USC.
    • Paper
        author    = {Jiang Wang and Yang Song and Thomas Leung and Chuck Rosenberg and Jingbin Wang and James Philbin and Bo Chen and Ying Wu},
        title     = {Learning Fine-grained Image Similarity with Deep Ranking},
        journal   = {CoRR},
        volume    = {abs/1404.4661},
        year      = {2014},
        url       = {http://arxiv.org/abs/1404.4661},
        archivePrefix = {arXiv},
        eprint    = {1404.4661},
        timestamp = {Tue, 08 Sep 2020 16:29:29 +0200},
        biburl    = {https://dblp.org/rec/journals/corr/WangSLRWPCW14.bib},
        bibsource = {dblp computer science bibliography, https://dblp.org}
  • YOLOv4

    Link: https://github.com/pjreddie/darknet

      author    = {Chien{-}Yao Wang and
                   Alexey Bochkovskiy and
                   Hong{-}Yuan Mark Liao},
      title     = {Scaled-YOLOv4: Scaling Cross Stage Partial Network},
      journal   = {CoRR},
      volume    = {abs/2011.08036},
      year      = {2020},
      url       = {https://arxiv.org/abs/2011.08036},
      archivePrefix = {arXiv},
      eprint    = {2011.08036},
      timestamp = {Wed, 18 Nov 2020 16:48:35 +0100},
      biburl    = {https://dblp.org/rec/journals/corr/abs-2011-08036.bib},
      bibsource = {dblp computer science bibliography, https://dblp.org}