If you liked the post, Share on Facebook, Tweet and Google Plus (use buttons above). You can also Subscribe to our feed via Email for free.
 |
Sixth Sense device | Make your own device easily
on 18 June 2012
Sixthsense is a wearable gestural interface device developed by Pranav Mistry, a PhD student from MIT that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.
Watch the following video of Pranav Mistry presenting sixth-sense device to get a clear idea about sixth sense device.
How to make your own sixth sense device?
You must be desperate to get your own sixth sense device. It is not available for sale so you cannot buy it from anywhere. The only option you have is to make it yourself. Yes, You can make it yourself!
SixthSense prototypes cost approximately $350 to build (not including the computer), the main cost being the micro-projector. Don't feel disappointed if you can't spend $350. You can still build the software and play with it for free.
Sixth sense device's software is now open source. I appreciate this move from Pranav Mistry, but the source code is full of bugs and exceptions as it is yet in beta version. I have edited the code to make sure it is compatible with both Visual Studio 2008 and Visual Studio 2010.
Compiling and running the software component
The sixth sense device runs on windows platform and majority of the code is written in C++ and C#. Make sure you have Visual Studio, Direct X, Adobe Flash player (active x) and MS Outlook installed on your computer.
You first need to download the following source code(zip file) and extract it. If you don't want to compile the software and save time then you can directly download the already compiled Sixth Sense software binary(.exe). If you download the exe file then skip steps 2 and 3.
Open WUW01.sln (which is present in the folder "code"). If you are using VS 2008 then instead of opening WUW.sln, open SixthSense\code\WUW01\WUW01.csproj
Press F5 to debug it. If you debug it successfully, a black screen appears with a little square box on the right just hover your mouse around it and you will find Option tabs such as Camera, Tokens and Apps.
Note: If you do not have the DIRECTX installed on your machine ( as stated in the requirements ) , it doesn't make any difference. Just comment out the following line nos. 41, 42 and 74 in the WUW.cs file. This works beacuse only a matrix class of the DIRECTX namespace is declared in the code but never used. So you can safely comment out these lines and remove the DIRECTX references.
I have solved various bugs and exceptions including the black box error. Also I have solved XamlParseException which gave following error:
Cannot create instance of 'WeatherReaderUI' defined in assembly 'WeatherReaderMVC, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
I have commented the line no. 111 and 1863(shown below) to solve the problem. Check if the program compiles and runs successfully in your system after removing the comment.
111: WPFControl_Weather01.UserControl1 Control_weather = new WPFControl_Weather01.UserControl1();
1863: elementHostWeather.Child = Control_weather;
Camera:
- Here you can select the camera you want to use from the drop down options.
- If you are having multiple cameras you can select from those option or if you have single camera it will automatically start capturing.
- You can also adjust your camera settings (Brightness,contrast,..etc) by clicking on "Adjust Camera Properties". (Note: Sometimes an exception occurs while clicking on Adjust Camera Properties button)
Tokens:
This version of code uses colored fingertips as markers. One need 4 different colors in order to run most of the Apps. I recommend to use Red, Yellow, Blue and Green (or colors that are different enough). Matt colored tape/objects works better as they have less reflective changes.
When you click on ‘Add Markers’, the screen starts blinking and after 4-5 secs it stops with a still picture so that you can add those markers on your hand as new markers. You can simply drag-select the colored marker on the screen by mouse cursor. Once you have all four markers (O, P, M, and N) added the software will start tracking the four markers. You can set the thresholds to better the tracking. You can save those markers and can reuse in subsequent executions, or can add new markers every time. Check this video.
Apps:
There are three ways to launch any application. Apps tab can let you choose the apps you want to load and run. One can also load apps from ‘Menu’. ‘Menu’ is triggered using Namaste gesture. Once on Menu screen you can choose the App by hand-click (Hand Click gesture works by moving the O marker and hiding P marker.
Test: You can see whether the four markers are being tracked and moving while you are moving those particular markers that is on the fingers.
Draw: The red marker (M) draws when the yellow marker (N) is hidden.When both the markers are visible you can move the mouse pointer to select different colours or eraser etc.
Photo: Make a rectangle with your pointer fingers and thumbs and when the gesture is recognized,the countdown starts Remove you hands from the camera frame to capture the desired photo. (The countdown lasts for 3 secs)
Gallery: Displays the photos that you have taken.
Mail: Displays a list of email from outlook (required that MS Outlook be running for this to work)
Weather: Displays a weather app that can be localised to your area.
Stock: Displays 16 stocks values real time, use the slider at the bottom of the projected display to set the time scale.
Effects: Play with beautiful wavy lines and explore your imagination.
Newspaper: It projects newspaper stories and related videos from Youtube or other web sites,directly onto the surface of the paper.
Book: For book-recognition,the program gets activated with a gesture, and the system snaps a photo of the book, compares it with book-cover images it finds online, computes a match, and retrieves and projects the ratings.
Clock: An Analog clock will be projected by doing a circle gesture.
Hardware Components
Now you have the needed software. Let's check out the various hardware components.
Camera
The camera is the main input device of the SixthSense system. The camera acts as a digital eye of the system. It basically captures the scene the user is looking at. The video stream captured by the camera is passed to computer which does the appropriate computer vision computation.
Suggested Product: Logitech QuickCam Pro for notebooks ($23)
Note:If you are just checking the software, your laptop's in-built web-camera will work.
Projector
The projector is the main output device of the SixthSense system. The projector unit used in prototype runs on a rechargeable battery. The major functions of the projector is to project graphical user interface of the selected application onto surfaces or walls in front.
Suggested Products: You can buy either laser (AAXA - $110, Microvision - $200) or L.E.D (3M MPro110 - $270) projectors.
Mirror
The mirror reflects the projection coming out from the projector and thus helps in projecting onto the desired locations on walls or surfaces. The user manually can change the tilt of the mirror to change the location of the projection. For example in application where we want the projection to go on the ground instead of the surface in front, we can change the tilt of the mirror to change the projection.
Suggested Product: Any 1”X1” first surface mirror
Microphone
The microphone is an optional component of the SixthSense. It is required when using a paper as a computing interface. When the user wants to use a sheet of paper as an interactive surface, he or she clips the microphone to the paper. The microphone attached this way captures the sound signals of user’s touching the paper. This data is passed to computing device for processing. Later, combined with the tracking information about user’s finger, the system is able to identify precise touch events on the paper. Here, the sound signal captured by the microphone provides time information whereas the camera performs tracking.
Suggested Product: Microphone of Logitech QuickCam pro for notebooks ($23)
Computer
Of course, you will need a computing device (i.e laptop/netbook). The camera, the projector and the microphone are connected to this device using wired or wireless connection.
Suggested Product: Any Windows computer
Now that you have all these pieces, you need a way to combine them. I recommend using Lego strips to form the base. The projector, camera, and mirror assembly can be directly put onto this base. You can also use Velcro to combine the products.
If you have any other queries or if you get any errors while compiling the code you can comment below.
39 comments:
i have the codes and everything but how do i install them to my camera?
SIR WAITING FOR YOUR REPLY...
Dear Mr.Akshay,
Sorry, I mistakenly deleted your previous comment. If I am not wrong, you want to develop project similar to Sixth Sense. It will be my pleasure to guide you but before that I would like to know more details about you. Please send any further communication on akashshastri.1@gmail.com
Regards.
Sir actually its my B.E. final year project, as I am team leader of my group.
I want to know that it will be implement or not..????
I want to talk to you more, but it not possible to talk in such way that we discuss., so i send you more details about me on your mail_id, so please check it earlier and help in my project (GUIDE ME PLZ)
WAITING FOR YOUR REPLY.
I was amazed at the TED presentation given on this device and its research and now I am too interested in producing a sixth sense device and developing the idea further. Instead of a laptop computer has anyone looked to utilise mobile tech i.e. Iphone/Android phone?
I have a spare mobile i will look to utilise to develop this concept with the aim of making the device more compact and simplier to use.
If anyone is interested in collaborating with me please get in touch. This is a great time to bring something to the general population where previously never possible increasing the human capability, amazing times.
Regards,
Carl
Sir I also need ur assistance as I am also working on this project. Please reply me ASAP. It would be very kind of you.
Contact me: krayservices101@gmail.com
My Name: Raj Sharma
I debugged the code, but it is not debugging and showing the following error(is not a valid Win32 application. (Exception from HRESULT: 0x800700C1). Please help me out i want to really use the technology.
sir good morning and i want to make this sixh sense as my final year. so please guide me how to build this project where can i get this hard ware components.
am very eager to know about this technology and want to develop the device if you can plz assit me
I'm really very interested in the project. Please help make it a success. I'll be very grateful if you lend your help and time for me and my team. Email id: jashrocker619@gmail.com
I have made this sixth sense equipment successfully... thank u mr.pranavMISTRY..
Sir what computing device can be used..
You can use a Windows laptop, netbook or a smartphone with high processing power.
How can I use the android mobile as a computing device? Please help me.
You can't use an android mobile as a computing device. It has to be a Windows running device. Though you can translate the C++/C# code into Java or use NDK to make it compatible to run with an Android device. Note: This will be really a lot of hard work.
hello sir,i want to make sixth sense device ,why WUW01.exe is not working.please suggest some idea
Would a Nokia Lumia 928 running windows phone 8 be a viable platform for running Sixth Sense?
can you please...give me assistance in making the code run in android device.? please...... reply to this message..
@dinakar I don't have much experience in android programming. Though I would like to recommend that writing your own code is much better and easier than translating sixth sense's c# code into Java.
Sir this technology is fantastic
is it possible to run the source code on a windows mobile or any device that runs on windows?
You need to compile the source code and then you can run it on any Windows OS. But the same executable file won't run on Windows mobile phone as Windows OS is not same as Windows Phone OS. Windows Phone OS requires a different type of executable because its applications have a very different life-cycle from desktop applications. The processors are also different and native binary code will not run cross CPU. Though if you are a developer, you create an app for Windows mobile phone OS by referring to the source code of Sixth Sense provided here.
sir iam a 13 year old guy truly inspried by this ihave all the things but i dnt know to combine cn i use my nokia lumia 520 (windows 8)instead of the camera pls email me and pls get me a vedio of combining these computer ,projector,andetc pls be fast sir bcz i want to gift it to an military officer his bday is 30thh december my email is llpcslakshmi@gmail.com
any one tried the compiled .exe file .? and if that works fine that can save a lot of debug time and efforts.
please share your experiences with that .exe file given as step number 1 under the -- "Compiling and running the software component" here on the page above.
Sir,i am second year engineering student..i want to build this project and work on it for next two years. i have downloaded the source code.bt problem m facing is on my one laptop its running fine and on another its showing system.badimageformat exception.please tell me solution. i also wanted to know about different gestures built in into code, for e.g. selecting any option.currently m not being able to change my color in drawing application. please tell me diffferent gestures in deep.please reply me.. i am damn serious about this project. my email id is:- sammyshinde14@gmail.com
hi sir,m second year engineering student.currently m working on this project but facing lot of difficulties in understanding different built in gestures.for e.g.m not being able to select any option from apps.i externally have to click on that button. so please give me brief information about that. one more question i have is,this code runs on ma 1 laptop totally fine.bt on another its showing system.BadImageFormatException
what should i do..?
where can i buy it? is there an official site
sir, i am trying to used it but it cannot work in win 7 64 bit.can u give me the 64 bit (.exe) file.
or tell me how i can change it to 64 bit.
i have installed turbo c++,dev c++ and visual basic software in my laptop but also i can't run it.
please help me sir. thank you.
send me your advice to ( nakulsahis@gmail.com).
So, should we need to carry our laptop everywhere? Reply anyone pls. roshanratnam91@gmail.com
Hello sir , can you consult me on how to make this device .
this is my final year physics project that I really need to pass to graduate.
i really hope you can help me .
hello sir , it is me again , i really need your help on this project .
can you assists me to make it happen ? I've already debug the code . All that I need is to connect them together but I don't know how. I really appreciate it if you can help me with this . I will be waiting for your reply
Hey Akash i almost buil this device the only problem im having is that its not projecting anything do i have to select projector or do anything to project things ?
one question
We should carry computer to wherever we use this device?
For example I want to use it in collage or outside home?
Any easier way? tablet connection?
Or just one we run the software and compile it. We don`t need PC anymore??????
can i use camera of mobile-phone in place of the above camera????
This project is using gesture as an input to the computing device/software through camera (which process and give certain output through projector)
Yes! You can use your android or iphone as a webcam.
Hello Sir,
I am work in this six sence device project,
1. Can you give me a circuit daigram of this full project,
2. how to connect this whole device to my android or iphone ?
#I AM A LEADER OF MY COLLEGE PROJECT
please replay me as soon as posible........
Sir can use HP stream 8 tablet
I would like to try and build the sixth sense my self but I am only 12 years old could you help me.
Post a Comment