Before we get the first VR application development, let's take a look at a few important concepts:
Build a scenario: Create a visual, interactive, script-driven virtual reality environment.
Stereo Rendering: Rendering the scene with two cameras to represent the user's left and right eyes, and then through the Oculus Rift head-mounted lens, the two images are rejoined together to form a clear and deep-visual scene.
Head Tracking: Change the position and steering of the camera in the virtual world by capturing the position and steering of the Oculus Rift head.
We need to write a lot of 3D manipulation code to express our ideas. We can render 3D views directly through OpenGL and DirectX, but this is a waste of time and is beyond the scope of this book. We use the unity3D game engine to do this. Unity is very suitable for quickly building VR content. The most important thing is that it is very easy to master.
Before delving into Unity, let's take a brief look at basic 3D graphics technology. If you are already familiar with 3D graphics programming, this part can be skipped directly.
3D graphics basic definition
Before we continue, let's take a look at the definition of 3D graphics. The following is a definition given by Wikipedia: 3D computer graphics (in co ntrast to 2D computer graphics) are graphics that use a three-dimensio nal representation of geometric data (often Cartesian) that is stored in the computer for the purposes of performing calculations and rendering 2D images. Such images may be stored for viewing later or displayed in real-time.
The above definition has three main parts: (1) All data is represented in a three-dimensional coordinate system. (2) They will eventually draw (render) on a two-dimensional map, where VR will be divided into left and right eyes on two maps. (3) The images are rendered in real time. When some animations or user operations cause 3D data changes, the images they render will be updated in real time. This update frequency must be undetectable. The last of the above three points is the key to building interactive applications. In fact, 3D graphics rendering technology is so important that it has created a multi-billion dollar market, and many large companies are focused on 3D real-time rendering technologies such as NVIDIA, ATI, Qualcomm, and others.
First, three-dimensional coordinate system
If you are familiar with two-dimensional coordinate systems, such as the Windows desktop application or the coordinate system used by IOS mobile applications, you must know the x and y axes. The 2D coordinates can represent the position of the subform or UI control. When the drawing API is called, the drawing points of the brush and the brush can be defined. Similar to two-dimensional coordinates, the three-dimensional coordinate system has just one more z-axis. This direction is used to describe the depth information (the distance of an object from the screen). If you already understand the concept of a two-dimensional coordinate system, then you can convert to a three-dimensional coordinate system. It's very simple.
Figure 3-1 is a schematic diagram of the coordinate system used in this book. Its x-axis is horizontal, the direction is left to right, the y-axis is vertical, the direction is down to the top, the z-axis passes through the screen, the direction is inside to outside, and These three axes are all perpendicular to each other. Some of the three-dimensional coordinate system has a vertical z-axis and the y-axis passes through the screen.
Figure 3-1
The coordinate system used by unity3d is the one shown above, except that its z-axis direction is outward. The figure shows the right-handed coordinate system, and the Unity3D is the left-handed coordinate system. It should be noted that OpengGL usually also uses the right-handed coordinate system.
Second, the grid, polygons, vertices
There are many ways to draw 3D graphics, the most used is to draw with a grid. A mesh consists of one or more polygons whose vertices are points in three-dimensional space and have three coordinate values: x, y, and z. Triangles and quadrilaterals are usually used in the grid, and these basic patches can be enclosed into a grid to form a model.
In Figure 3-2, there is a three-dimensional grid. The black lines are the sides of the quadrilateral. These quadrilaterals shape the shape of a human face. Of course, these black lines are not visible in the final rendered graphic. The node coordinates of the grid only represent the shape of the model. The color and illumination of the mesh surface are represented by additional attributes, which we will introduce later.
Figure 3-2
Third, material, texture, lighting
In addition to the x, y, and z coordinates, the surface of the mesh is represented by additional attributes. Surface properties can be very simple in monochrome, or in complex ways, such as how reflective it is or whether it looks shiny. The grid surface can also use one or more bitmaps, one is called a texture, and many of us are called atlases. The texture can be a text effect (such as a pattern on a T-shirt) or a complex rough or rainbow effect. Most graphics systems use a uniform representation of the surface properties of the mesh, and the resulting effect of the material is affected by the lighting in the environment.
The material used in the model in Figure 3-2 is dark purple, and the surface lighting effect is illuminated by the left side, which we can see through the dark part of the right side.
Fourth, the transformation matrix
The three-dimensional spatial position of the model mesh is determined by their vertex coordinates. If you want to move the model position each time, you must change the vertex coordinates of each mesh in turn. This will be a very headache, if you encounter it. It needs to be animated to be worse. In order to solve this problem, most 3D systems provide conversion operations. The principle of this operation is to move the mesh as a whole, so that there is a relative transformation between the mesh and the world coordinates, without changing the coordinate values ​​of each vertex. Among them, the conversion operations include: movement, rotation, and scaling. These operations are all directed to the grid relative to the world coordinate system, rather than specific vertices.
The conversion operation is shown in Figure 3-3. There are three cubes in the graph. Each cube consists of a cube mesh. They all contain the same vertices. We don't need to change these when we move, rotate, and zoom. The coordinate value of the vertex, but a transformation operation is given to the cube mesh. The red cube on the left has moved 4 units to the right ([-4,0,0] operation), and then rotated relative to the x and y axes (note here, the unit of our angle here is radians, ie one The radians are equal to 360 degrees divided by 2*PI). The blue cube on the right moves 4 units to the right and then magnifies 1.5 times in all three directions. The middle green cube is the initial position.
Figure 3-3
We use a matrix to represent the conversion operation. This matrix holds an array. Through this array, some mathematical calculations can be used to obtain the coordinates of the transformed vertex. Most of the conversion matrices are represented by a 4*4 array containing 4 rows, 4 columns, and a total of 16 numbers. Figure 3-4 is a schematic diagram of a 4*4 array, where m12, m13, m14 are used to operate the movement, m0, m5, m10 are used to operate the scaling, m1 and m2, m4 and m6, m8 and m9 are used to operate relative The rotation of the x, y, and z axes, the transformation matrix multiplied by the vertex coordinates is the coordinate after the transformation.
Figure 3-4
If you are a linear algebra geek like me, I can understand it this way. If you are not familiar with linear algebra, it doesn't matter. Unity3D and other tools have already encapsulated these operations. We only need to call them correctly. The API is fine, but it's always good to know the underlying computing process for these operations.
Five, camera, perspective, viewport, projection
Rendered scenes all require a view that can be viewed by the user. We usually use cameras to provide this in 3D scenes. The camera has a position and orientation relative to the scene. Just like the camera in our life, it also provides a perspective view, which can have a near and far effect. The camera will eventually render the 3D scene into a two-dimensional image that we can observe through its viewport.
The camera deals with two matrices in the processing. The first one is the linear transformation matrix (the previous section we call the transformation matrix), which is responsible for defining the position and orientation of the scene object, and the second is the projection matrix, which is responsible for the 3D scene. The object is projected into a 2D viewport. Of course, the specific details require too much mathematical theory, so unity3D has already packaged these, and our developers only need to simply "target and launch".
Figure 3-5 shows the camera core concept viewport and projection. In the lower left corner of the figure, the big eye represents the position of our camera, the red x axis represents the camera's orientation, and the two blue squares. It is the object to be observed. The green and red rectangles represent the near and far cut surfaces respectively (the object between the two cut surfaces can be seen, and the area between them is called the visible flat cut vertebra), and the near cut surface is Our viewport, the image projected on the viewport is the image we actually see.
Figure 3-5
The camera is very powerful, it allows the observer to observe a 3D scene very realistically. In addition, it provides powerful support for animated scenes. By moving the camera, you can create a dynamic narrative scene like a movie. Of course, for VR needs us to not move the camera at will. If we want to move the camera, we need to consider it in many ways. It is best to let the user fully control the camera, so that the user will have an immersive feeling.
Sixth, stereo rendering
In virtual reality , 3D image rendering is a big problem. However, nowadays it has encountered the problem of how to deal with the camera. The camera itself needs to process the transformation matrix and the projection matrix. The awkward thing is that we still need to process twice in VR (each Eye treatment once). But the situation is not so bad, we have several ways to deal with the camera, here's a simple way to introduce:
Create a main camera: the application controls the only one main camera, all the logic animations, etc. are only for the main camera, so that we become simple and unified when dealing with the camera to interact with other objects, in addition to pay attention to the main camera does not participate Real rendering. Another advantage of the main camera is that it can be switched between a dual stereo view and a traditional single view.
Rendering with two cameras: In addition to the main camera, the VR program requires two additional cameras to actually handle the rendering. These two cameras need to be in line with the position and orientation of the main camera, except that they are shifted a little bit left to right to simulate our interpupillary distance.
Render to two viewports: The left and right eye rendering cameras created by the VR app, each with a viewport width that is half the width of the screen and a height that is the height of the screen. Each camera processes a graphic with a special projection matrix that can handle the anti-distortion problem (this algorithm is available in the Oculus SDK).
At this point, we have already briefly understood the image processing process of VR. It is very simple here. If you want to go deeper, every point can be introduced with a whole book. As your program gets bigger and bigger, you need to deal with a lot of underlying problems. It's best to make a game engine for these problems. Of course, if you are very NB, you can write an engine yourself, but if you follow I also want to focus on the application layer, so it is best to use a ready-made engine. There are many good engines on the market, including the unity3d game engine to be introduced in the next section.
Tv Speaker,Wireless Soundbar,Single Soundbar Speaker,Soundbar With Wireless Subwoofer
Newmax Electronics Co.,LTD , https://www.fspeaker.com