Development of Smart Projector

Chapter 1: Introduction

1.1             Problem Summary and Introduction

Our institute owns 3 ordinary projectors (Projectors with no Wi-Fi) and whenever professors require it, they have to book place and time by manually going to Admin and writing down the details. The day the professor gets his/her hands on the projector, it takes about 15-20 minutes just to set it up and about 5-6 minutes to get it started. This is very time consuming and complicated, too. If the professor brings a laptop with them then they have to use external adapters and dongles to connect to the VGA port in the backside of the projector. And mobility is zero, you cannot connect your smartphone with the projector. The main problem is the cheaper projector consists of less quality while projectors with good quality are expensive. So, we are trying to solve three major problems:

1)      Making a projector which connects with smartphones and laptops seamlessly

2)      Making a projector which has better interaction features

3)      Making it all in an affordable and cheaper amount

1.2             Aim and Objective

The main objective of this project is to innovate the conventional strategy which is used to educate students. In the conventional strategy, the regular whiteboards and blackboards are used by the professors and when it comes to presenting digital media like presentations, spreadsheets, etc. projector has to be carried around. So, the objective of this project is to develop a smart projector that helps teachers, professors, and anyone to illustrate things better and in the much interactive way.

1.3             Problem Specification

The main problems faced by normal projectors are listed below:

1)      Conventional Projectors can’t connect to smartphones or tablets

2)      Conventional Projectors need dongles and cables to connect to various laptops or PCs

3)      Conventional Projectors are heavy to lift them around so it has no portability

4)      Conventional Projectors take much time to connect to a device. For e.g. to connect a MacBook Pro to the conventional projector it takes about 15-20 minutes.

5)      Projectors which are cheap in price; don’t serve better quality. (for e.g. mostly cheap Chinese projectors which complicates user in downloading faulty and unsafe apps on their smartphones)

6)      Projectors with many features and better quality are very expensive.

1.4             Brief Literature Review and Prior Art Search (PAS)

1.4.1 Patent Search

1)      Digital Projector (Ordamo Ltd. – GB2537225A)

Restaurants and Cafes always have the most common food ordering system from a very long time. And as per the general fact, this has been always the same way, but bot work efficiency is way better than that of a human. Digital Projector makes food ordering intuitive and interactive by using projectors showing the food menu and IR camera sensing the gestures directly on the table.

2)      Mobile terminal and method of controlling the same (LG Electronics Inc. – US20110267316A1)

We use Mobile Phones in our everyday lives and we have to carry them around for our everyday tasks but the problem is that they are bulky and they have a fixed screen size. If one wants to draw or doodle something, they have to buy a stylus or pen and maybe even afford a tablet for better productivity. With this invention, people just have to carry this digital pen with them which lets them play music, send or receive messages, set an alarm or call someone straight from the pen.

3)      Programmable music play system based on Raspberry Pi (Tongji Univ. – CN107610683A)

Compared with the prior art, the system is programmable, is portable, greatly reduces the cost in learning the related knowledge of the music and musical instruments, and reduces the performing difficulty of the musical instruments. This can be used to replace Musical Instruments in real life.

4)      Digital teaching all-in-one machine (Shenzhen Haiya Tech. Dept. Co. Ltd. – CN203102627U)

The digital teaching all-in-one machine is a type of device which consist a system which enhances the education by digitizing it into a much flexible and easily controlled way.

5)      Notebook with mini-projector (Acer Inc. – US20090046035A1)

This patent included a notebook with a mini-projector enclosed in it. The mini-projector is at the top which can project the images and media directly from the notebook which makes it a super portable and convenient device for situations like business meetings and project meetups.

1.4.2 Market Search

Figure 1.4.1 from Google Trends shows the interest over time for ‘projector’ keyword which is used by various users from Gujarat on Google Search. The latest Aug 26, 2018 data shows a score of 78 which is more than a half popular.

[Figure 1.4.1]

Below Figure 1.4.2 shows the popular ads shown and clicked by user for the keyword ‘projectors’ which has over 100K to 1M average monthly searches.

[Figure 1.4.2]

1.5             Plan of their work

The invention is about a Digital Projector which is also applicable in displaying food directly onto the table while ordering. The components are projector, video camera, cooling system, raspberry pi, and an access point. There is a projector which projects the images on the table. Video Camera tracks the motion of user’s touch. There is an access point that manages the projector and video camera. Raspberry Pi is embedded which helps with the remote management of system. Which makes an interactive food ordering system and which can be implemented to any restaurant.

The second invention is about the Mobile Terminal and Method of controlling the same. In this, the mobile pen uses the sensing unit to track user’s input from the projector’s projected display, which shows a great example how mobility could affect and reduce the size of overall design. It also gives an idea that how sensing unit can help user interact with the device so rich and intuitive.

The third invention that got our intention is Programmable music play system with Raspberry Pi. The invention relates to a programmable music play system based on the Raspberry Pi, and the system comprises an infrared ray output module, a camera module, a Raspberry Pi development board, a keyboard pattern projection module, a music output module and a power module [15]. The Raspberry Pi development board carries out the image processing, and recognizes the control gesture or key operation of a user on the keyboard pattern, and controls the music output module to play corresponding music [15].

The fourth invention is similar to our project which is Digital teaching all-in-one. The utility model includes an electronic whiteboard with a signal receiver, an electronic signal sending pen, support arms, loud speakers, camera and a centre control remote receiver which makes it a digital teaching machine for everyday classrooms [4]. It has a flexible control way, occupies small space, high teaching quality and efficiency making it a perfect device for teachers in the classroom. Digital teaching all-in-one machine digitizes the everyday classroom and makes it even more interactive and intuitive.

The fifth and last invention we got from searching through the web is Notebook with mini-projector by Acer Inc. A notebook with a mini-projector is disclosed [12]. The notebook consists a mini-projector at the top that can project images which enhances the usage and convenience.

1.6             Materials/Tools Required

The following items are required to build up the Smart Projector:

1)      Projector Lens and Circuit

2)      Raspberry Pi 3 Model B + SD Card

3)      IR tracking Camera (x1) [May or may not require]

4)      IR LEDs (x5) [May or may not require]

5)      Wireless Adapter (x1)

6)      Speaker (x1)

7)      Keyboard and Mouse

8)      A custom casing to enclose all these items

Chapter 2: Design

2.1 Ideation Canvas

First we started with people, where we simply thought about the people for whom we want to solve the problem. For our Product we listed the people like Businessmen, Employee, Student, and Professors Etc. Then listed out what every people does in every segment of activity. Then noted out context/location/situation and finally for possible solutions we contacted members of other groups. Figure 2.1.1 shows the Ideation Canvas

2.2 Product Development Canvas

Figure 2.2.1 shows the Product Development Canvas. In which, we have mentioned purpose of our product, functions, features, revalidation, product experience. Purpose of our product was to change people’s perspective for how ordinary projectors can become more useful and interactive with a little upgrade. We also visited other teams as a customer for User revalidation.

2.3 AEIOU Summary

There are four parts in AEIOU Summary which are Activity, Environment, Interaction, Object and User which are detailed in Figure 2.3.1. In the Activity, we have considered activities which are presentation, interaction, drawing and objects as smartphone, projector and Raspberry Pi. In the Environment, we have considered quiet because most classes or meetings are taken place in a calm or quiet environment. Main interactions which are possible for this project is Student to Student, Student to Professor, Employee to Employee, Businessman to Businessman. And, users are student, professor, employee and businessman.

2.4 Empathy Canvas

We have considered possible users which are affected by this project which are students, professors, employees and businessmen. Stakeholder is the person for whom this project would be most helpful as we listed out professors, businessmen and employees. We noted out activities which are presentation, casting, teaching, interaction, drawing, elaboration. And at the end we wrote happy and sad stories targeting these stakeholders. Figure 2.4.1 shows Empathy Canvas.

[Figure 2.1.1]

[Figure 2.2.1]

[Figure 2.3.1]

[Figure 2.4.1]

Chapter 3: Implementation

The Project includes a module which is Raspberry Pi 3 Model B. Our main goal is to make the projector able to connect with smartphones seamlessly and of course wireless, too. In order to do that we have to show the content to the projector from a mobile device. We studied about various types of mobile casting which are DLNA (Digital Living Network Alliance) [5] and Miracast. In which DLNA is basically casting the content to the device over Wi-Fi. For e.g. The Cast feature available in most Android smartphones which lets you play your YouTube videos directly on your Android TV or Chromecast devices is DLNA based. Miracast is actually reflecting your device’s screen content to another device on which it is being cast that is why in other way Miracast is also known as “Mirror Casting”. It was clear that we have to implement Miracast for our project.

Now, we knew that we have to implement Miracast in our project so we searched on the internet about implementing Miracast on Raspberry Pi 3. The interesting fact is that Raspberry Pi 3 Model B has an on-board Wi-Fi chip which is of 2.4 GHz so it may consist slower data rate but it has farther range. And it does support Wi-Fi Direct which basically is used for Casting. Wi-Fi Direct is different from Normal Wi-Fi Connection because it is a P2P (Peer-to-Peer) connection which lets two peers establish a connection with each other which is more secure and faster than a regular Wi-Fi connection.

[Figure 3.1.1]

Figure 3.1.1 shows the Wi-Fi chip on the Raspberry Pi model B. We tried experimenting with it and we found an online project for Miracast implementation on GitHub which is Miraclecast [10]. We tried contributing the project which could be useful for the project but at the end system logs suggested bugs which weren’t possible to fix until system patches it up with an update [11]. It was a failure.

After that, we still had to find a way to screencast the smartphone so we tried different GitHub repos but none of them worked and we failed every time. Here are the attempts and the reason why they failed for us:

[Table 3.1.1]

After the failures, we finally created an app that opens up a port on Android Device and Raspberry Pi discovers the device which are in the same Wi-Fi network and screencasts the content to the Raspberry Pi. This was implemented with Google’s MediaProjection Library [8]. It is quick and the frame rate is better than the previous attempts. Scan the QR code to watch the video that shows the smartphone screencast with the Raspberry Pi.

We also implemented Apple’s Airplay for iPhones and for laptops we used Remmina [18] to connect to laptops wirelessly via Remote Desktop Connection.

Now, all we had to do was to create an app for the Raspberry Pi which shows user to select which type of device is being used. Basically most applications are built upon the x86 and x64 architecture. We needed an app that could run in the Raspberry Pi with the backbone of its ARM processor architecture. So, we developed an electron JS [7] application which can run on different platforms as shown in below Figure 3.1.2. The app didn’t run as expected on Raspbian.

[Figure 3.1.2]

We again developed a Windows Forms App with the help of C# and Project Mono [16] which ran perfectly on Raspbian as shown in Figure 3.1.3.

[Figure 3.1.3]

This app will start up as the Raspberry Pi will boot up with the Projector. And, by clicking the icon the respective script will connect the device.

Chapter 4: Summary

4.1 Summary of Results

We implemented Miracast for Android Devices via the help of MediaProjection APIs [8]. Thus, it was clear that giving too much reliability on Raspberry Pi’s Wi-Fi Direct may lead to some error as it was already generating many errors at the time of configuration. We also developed an app for Raspberry Pi which lets user select which type of device the user wants to connect to which makes it a singleton application for connecting any type of devices on the go. The Raspberry Pi is connected to the Projector Module which works as a Display Output.

4.2 Advantages of Project

Conventional Projectors are used in most classrooms are not portable means they can’t be carried around easily. This project makes the projector portable which encloses the projector module and the processing unit which is a single Raspberry Pi into a single casing.

To connect to a conventional projector dongles and various types of cables are required to connect to different types of devices. Smart Projector makes the connection to various devices wireless, so no cables are required.

Conventional Projector can’t connect to various smart devices like android smartphones or iPhones, but smart projector makes it easier to connect to various devices.

Conventional Projectors takes at least 15-20 minutes to connect to a smart device for e.g. a MacBook Pro will take a lot time to connect to a conventional projector. Smart Projector doesn’t require dongles or cables because it connects devices over Wi-Fi and it takes just 5-6 minutes to start up and the connection is established instantly.

4.3 Scope of future work

As the name suggests, “Smart” Projector should consist a feature or two that makes it smarter. Conventional Projectors uses keyboard and mouse to interact with the system which makes it even less portable. We want to develop an interaction module which lets users interact with the projector directly touching the projected display on the surface making it a perfect tool for professors, teachers and presenters. As it supports touch interaction, users can draw anything on it. Classrooms can be more interactive and intuitive by using the projector for illustrations.

We also want to develop a password protected access point module which is initiating a single access point network for the connection between the device which’ll connect to the Projector and the Projector itself making it a fast and secure channel for transferring data. This will make the Projector requiring no common external Wi-Fi connection for making it work.

4.4 Benchmarking with existing solutions

We searched on the internet for products or devices which matches our project and here are some of them which caught our interest.

1)      EGATE Wi-Fi Projector [6]

Similar Features: Portable, Supports Miracast [9], Airplay, Effective Cost

Limitations:

• Wi-Fi disconnects after every 3 seconds.
• Users have to download various malicious apps to Miracast their smartphone same for iOS devices.
• The projector display quality is not good as it is promised to the consumer.

2)      unic UC46+ [19]

Similar Features: Portable, Support Wi-Fi connectivity with smartphones, Effective cost

Limitations:

• Projector display quality is way too cheap. Many of the customers who’ve bought the product had problems with the display.
• Wi-Fi connectivity is defective and slow.
• It comes with a USB port which doesn’t even works sometimes.

3)      Airtame [1]

Similar Features: Ultra-Portable, Wireless Connectivity (Supports Miracast [9], Airplay)

Limitations:

• A Projector is required to make it work to make it similar to
• It is way too expensive.

References

[1] Airtame Wireless Adapter; www.airtame.com

[2] Android Transporter; https://github.com/esrlabs/AndroidTransporterPlayer

[3] Digital Projector; https://patents.google.com/patent/GB2537225A/

[4] Digital teaching all-in-one machine; https://patents.google.com/patent/CN203102627U/

[5] DLNA; https://www.dlna.org/

[6] EGATE Wi-Fi Projector; https://www.amazon.in/EGATE-i9-Android-WiFi-Projector/dp/B0711F318V/

[7] Electron JS; https://electronjs.org/

[8] MediaProjection API; https://developer.android.com/reference/android/media/projection/MediaProjection

[9] Miracast; https://www.wi-fi.org/discover-wi-fi/miracast

[10] Miraclecast; https://www.github.com/albfan/miraclecast

[11] Miraclecast Issue about Miracast on Raspberry Pi; https://github.com/albfan/miraclecast/issues/265

[12] Mobile Terminal and Method of controlling the same; https://patentimages.storage.googleapis.com/9d/a3/4a/044aca4b73edbb/US20110267316A1.pdf

[13] Notebook with mini-projector; https://patents.google.com/patent/US20090046035A1/

[14] PiCast 3; https://github.com/lanceseidman/PiCAST

[15] Programmable Music Play System based on Raspberry Pi; https://patents.google.com/patent/CN107610683A/

[16] Project Mono; https://www.mono-project.com/

[17] RaspiCast; https://play.google.com/store/apps/details?id=at.huber.raspicast&hl=en_IN

[18] Remmina; https://remmina.org/

[19] Unic UC 46+; https://www.amazon.in/UNIC-UC46-Upgraded-Miracast-Projector/dp/B01I0VU9OW