opensse 0,0,0,0,2,1,1,0 travis-ci

Open Sketch Search Engine- 3D object retrieval based on sketch image as input

2 years after Apache-2.0

OpenSSE: Open Sketch Search Engine

![Gitter](https://badges.gitter.im/Join Chat.svg)

Hello everybody!

I'm very interested in Mathias Eitz's works on SIGGRAPH 2012, so I developed OpenSSE, Just for fun.

This is an open source sketch search engine for 3D object retrieval based on sketch image as input. In fact, it can retrieve more, such as images, videos, point cloud models and volume models, you just need get their line drawing images than I know you can.

In order to clearly show how to work, I used single thread and ascii encoding for file storage.

Zhang Dongdong

Demo - SketchSearchDemo

This demo deals with mesh models. See more search results, click demo videos.

Sketch search demo

Demo - SketchRecognize

Try via web

See more search results, click demo videos.

demo video

How to use

Use OpenSSE library

You just need include one line in your .pro file.

include(../../opensse.pri)

Use tools

There are a series of tools for you under opensse/tools/. When compiled, those tools are under build-xxx-Release/bin.

Use gui

This is gui demo for you under opensse/gui.

How to train data

Database

Database in my demo comes from SHREC 2012, download 3D target dataset(~112MB). We use models under directory Watertight_dataset/Extended.

$ mv /Users/zdd/Download/Watertight_dataset/Extended /Users/zdd/Database/SHREC12/

Get line drawing views

Using my another project, you will get 102 views for each model.

Generate vocabulary

  • Step 1: Generate line draing images filelist
    $ ./generate_filelist -d /Users/zdd/Database/SHREC12/ -f "*.jpg" -o /Users/zdd/Database/SHREC12/2012_filelist
  • Step 2: Extract descriptors
    $ ./extract_descriptors -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -o /Users/zdd/Database/SHREC12/2012_

    you will gain two files: 2012_features and 2012_keypoints.

  • Step 3: Generate vocabulary (optional)
    $ ./generate_vocabulary -f /Users/zdd/Database/SHREC12/2012_features -n 1000 -o /Users/zdd/Database/SHREC12/vocabulary

    It will take you some times, but only need run once. or use my vocabulary under ../opensse/data/.

Create inverted index file

  • Step 4: Quantize feature
    $ ./quantize -v /Users/zdd/Database/SHREC12/vocabulary -f /Users/zdd/Database/SHREC12/2012_features -o /Users/zdd/Database/SHREC12/2012_samples

When dealing with large scale database, we can directly generate samples using extract_quantize tool to reduce disk usage without feature files.

$ ./extract_and_quantize -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -v /Users/zdd/Database/SHREC12/2012_filelist -v /Users/zdd/Database/SHREC12/vocabulary -o /Users/zdd/Database/SHREC12/2012_samples
  • Step 5: Create inverted index file
    $ ./create_index -s /Users/zdd/Database/SHREC12/2012_samples -o /Users/zdd/Database/SHREC12/2012_index_file

    Now all data are ready.

Test OpenSSE

Command line

You can test opensse through command line

$ ./sketch_search -i /Users/zdd/Database/SHREC12/2012_index_file -v /Users/zdd/Database/SHREC12/vocabulary -d /Users/zdd/Database/SHREC12/ -f /Users/zdd/Database/SHREC12/2012_filelist -n 10 -o output

You will get a interactive interface, like:

>> open sketch search :
>> input absolute path, like "/Users/zdd/zddhub.png"
>> input q exit
>> good luck!
>> /Users/zdd/Database/SHREC12/Extended/D00057view/1.jpg
0.985509 /Users/zdd/Database/SHREC12/Extended/D00057view/1.jpg
0.953196 /Users/zdd/Database/SHREC12/Extended/D00050view/1.jpg
0.938705 /Users/zdd/Database/SHREC12/Extended/D00041view/1.jpg
0.856004 /Users/zdd/Database/SHREC12/Extended/D00050view/2.jpg
0.852683 /Users/zdd/Database/SHREC12/Extended/D00057view/2.jpg
0.849087 /Users/zdd/Database/SHREC12/Extended/D00041view/2.jpg
0.823276 /Users/zdd/Database/SHREC12/Extended/D00041view/0.jpg
0.821133 /Users/zdd/Database/SHREC12/Extended/D00050view/10.jpg
0.811917 /Users/zdd/Database/SHREC12/Extended/D00057view/10.jpg
0.810096 /Users/zdd/Database/SHREC12/Extended/D00050view/0.jpg
>> q

Gui Demo - SketchSearchDemo

You must config params.json file using your data path, like:

{
    "searcher":
    {
        "indexfile": "/Users/zdd/Database/SHREC12/2012_index_file",
        "filelist": "/Users/zdd/Database/SHREC12/2012_filelist",
        "vocabulary": "/Users/zdd/Database/SHREC12/vocabulary",
        "rootdir": "/Users/zdd/Database/SHREC12",
        "results_num": "12",
        "views_num": "102"
    }
}

You'd better to use the absolute path. And then, set params.json in ../opensse/gui/SketchSearchDemo/mainwindow.cpp:

boost::property_tree::read_json("/Users/zdd/Database/SHREC12/params.json", params); 

Compiled and You will get my demo. Good luck!

Notice: To make sure line drawing images correspond to 3d models, please copy ../opensse/data/view/ to you execute file path.

By the way, press 'C' in sketch pad, You will clear the sketch and draw next one.

How to compile

OpenSSE uses Qt 5.1+, OpenCV 2.4.7+ and boost 1.55.0, it's a cross-platform library.

You need config INCLUDEPATH and LIBS in ../opensse/opensse.pri file related your path.

Now compile it, so easy!

Docker support

Someone still meets compile issue, so I add docker support. Please compare your environment with docker when you meet compile issue.

Run on mac:

# Install socat and xquartz if you want to run GUI demo.
brew install socat
brew cask install xquartz

# Start XQuartz
open -a XQuartz

# Expose local xquartz socket via socat on a TCP port
socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"

# in another window under our opensse repo folder.
docker build -t opensse .  # will takes ~15 minutes to build
docker run -it opensse

# # or use inet ip, like:
# docker run -it -e DISPLAY=$(ipconfig getifaddr en0):0 opensse

# Run command lind demo
docker run -it opensse /bin/bash
cd ~/opensse/build/tools/bin
# You will find all opensse tools

# If you don't want to compile code, You can use zddhub/opensse-demo to run demo:
docker pull zddhub/opensse-demo
docker run -it zddhub/opensse-demo

Docker images: zddhub/opensse-demo

Evaluation

You can use dataset and evaluation methodology to evaluate opensse on SHREC 2012, SHREC 2013, SHREC 2014.

You can generate data use test_search tool in tests/test_search.

Usage:
test_search -p paramsfile -i searchfilelist -r searchfileroot -o resultdir

Troubleshooting

  • Could not resolve SDK path for 'macosx10.9'

    There is a workaround: Navigate to where you installed Qt (default /Users/your username/Qt) using finder Go to the subdirectory .../5.3/clang_64/mkspecs directory

    Open the file called qdevice.pri with a text editor Change the line !host_build:QMAKE_MAC_SDK = macosx10.9 to:

    • !host_build:QMAKE_MAC_SDK = macosx10.10 if you are on OS X 10.10 (Yosemit
    • !host_build:QMAKE_MAC_SDK = macosx10.11 if you are on OS X 10.11 (EI Captian))

To do list

  • A sketch collect, share and evaluate platform
  • A better search framework
  • A new rank algorithm

Thanks

License

Code is under the Apache License, Version 2.0.

Donate

I accept tips through Alipay. Scan below qr code to donate:

Alipay.

Contact me

If you have any question or idea, please email to me. Or search 'zddhub' to find me in social networking platform.

Related Repositories

opensr

opensr

Open Sketch Recognize ...


Top Contributors

zddhub ferambot gitter-badger

Releases

-   v0.1.0 zip tar