Now we have covered the simulated reality engine and simulation, it is time to take a closer look at the systems that enable the simulated reality engine to remain operational. A true simulation should not be designed as a technical system in isolation but treated holistically as part of a broader social-economical, cultural system to fully realize the benefits and avoid exclusion, exploitation or other negative side effects.. The primary supporting system must implement the requirements we placed on sustainability, law, politics, economy, culture and values and consists of the following sub-systems or services that we discuss next:
- Technology development: The simulated reality software, hardware and networking technology continuously needs to be improved to add new functionality and fix bugs
- Electronics: The consumer devices to render the simulated experience and provide input to the simulation must be manufactured and distributed through supply and retail chains billions of users globally.
- Hosting: The simulation needs to run on a hardware and software cloud infrastructure and be closely monitored and maintained to ensure billions of people can interact with the simulation
- Telecommunications: The simulated reality needs fast high bandwith networks to provide a reliable Quality of Service.
- Energy supply: The simulated reality needs a continuous supply of electricity to perform the vast amount of computations.
- Material supply: A global supply chain of raw commodities and semi-finished goods is necessary to sustain the hosting and electronics systems and recycle components that have broken down or become obsolete.
- Finance and trade: Money must be able to flow in and out the simulated reality assets and ecommerce to buy and sell virtual assets.
- Law and Insurance: Safety, privacy, security and other risks users of the simulated reality must be mitigated in laws and regulations that are enforced and protect users.
- Governance: People must be able to rely on a democratic simulation governance to ensure system functionality that is not democratically agreed or meets laws and regulations is not implemented.
- Education: The best defense against misuse of the true simulation is a knowledgeable society and education system
Technology development
The simulated reality industry will be both larger and more influential than Big Tech is in the 2020s because the global economy will be the simulation. The power and influence of the companies that build and operate the simulation must be regulated by a strong democratic government system to ensure healthy collaboration and competition in a global simulated reality eco-system. This global core simulation technology eco-system includes the semiconductor, computer hardware, consumer electronics, networking, software, cloud service providing, game and app provider eco-systems and provides work to millions of people to improve the underlying simulated reality technology, but this is dwarfed by the billions of people who live, work and play in the experience society.
Not only the competition in the eco-system itself but also the development of technology will be more controlled by democratic governments and online fora as people have become aware, educated and experienced in the simulation where most of their life happens. Instead of society following technology development and being pushed by technology companies in certain directions, technology companies will follow the people and society who co-create the future direction with them. For example, the technology development system will be directed to build-in physical safety time limits for kids and adults for (older generation) virtual reality headsets to prevent damage to their eyes https://www.bbc.com/news/technology-52992675.
Electronics
To interact with the simulated reality, people will need human input and output devices. These electronics devices will provide the true immersion visual, audio, touch, scent and taste experience in our homes and enable us to talk, move, point and behave in the simulaton in such a way that we almost forget we are in the simulation.
The electronics system enables the continuous supply of new, repaired or refurbished devices through global supply and distribution chains into people’s homes. Like today when we order a game console or headset online or in the physical we will use e-commerce to replace our omni-directional movement device or scent brain computer interface with a newer, better version. For people with less technical skills or who do not have time, electrical installers will help users to install and configure their devices to ensure a properly functioning electronics system at their home.
As the simulation system will support multiple hardware configurations and even modular hardware changes [Modular smartphone – Wikipedia] the electronics system reduces the need to change devices frequently and works with legacy hardware to have a good simulation experience even on older less capable devices. When devices are broken, need repair or have become obsolete, the electronics systems ensures via a reverse supply chain that the devices are recycled by the material supply system (2.2..
Hosting
Today companies such as Amazon, Microsoft and IBM offer hosting services to support the ecommerce, streaming, entertainment and thousands of other internet applications. Google Stadia is a cloud-based gaming service that offers 4K and 60 frame per seconds on smartphones, tablets, TV, and computers that support the Google operating environment. Stadia: één centrale plek, voor hoe je ook speelt (google.com)
The simulated reality system itself cannot run on the electronics system in our homes, even if you had the latest hardware installed: The amount of computation is orders of magnitude higher than all the cloud service providers in the world provide today. Data needed – how many servers, how much computing cycles? The computation and storage of the simulated reality therefore must be hosted in the cloud and streamed to the electronics system. In the cloud we can make a separation between the computing cores that calculate the next state of the simulated universe and the computing edge computer that receive input and render output to the electronics system.
These data centers must be built in close proximity (within few hundred kilometers) to large population areas to ensure low latency and fast response times to user input provided because light travels at very high but fixed speed. As land is expensive and scarce near population centers and could be used better for agriculture or left to nature, we have two options, submerge the cloud under water near the coast or elevate the cloud to low Earth orbit (100-1000 kilometers above our head)
Microsoft reported positive results from trials to submerge containers with computers in the US, leading to lower energy footprints because of passive cooling with sea water and space reduction by packing servers closer together. https://www.allaboutcircuits.com/news/microsoft-brings-sea-to-servers-with-two-phase-liquid-immersion-cooling/. Excessive heat from the submerged computing clusters could be reused for aqua farming. Finnish Wartsila for example powers a Belize shrimp farm using the heat from a nearby power supply Wärtsilä power for Belize shrimp farming (wartsila.com). What is unclear is the effect of massive computing infrastructure under water to global warming of Earth’s oceans or biodiversity in coastal areas that are important to marine life. Calculation needed
Elevating the cloud to low Earth orbit would have the advantage that the infrastructure will not have a footprint on planet Earth and therefore will not take up valuable space or have potential negative effects on the climate. Launching and maintaining computing sattelites in LOE has its own challenges. First, even with reusable space rockets lifting payload into space is expensive. Today according to NASA is costs $10,000 to put a pound of payload in Earth orbit. NASA’s goal is to reduce the cost to space to hundreds of dollars per pound within 25 years and tens of dollars per pound within 40 years. NASA – Advanced Space Transportation Program fact sheet. Secondly, radiation is a problem for computers on spacecraft. Modern CPUs are very sensitive to radiation because billions of transistors are crammed onto a tiny surface. on Earth this is not an issue, but in space many errors are caused during calculations. To circumvent this problem, chipsets must be etched on other materials than silicon and have less transistors (less computation), all this comes at a higher price because space is a nice market for semiconductor companies. https://arstechnica.com/science/2019/11/space-grade-cpus-how-do-you-send-more-computing-power-into-space/. For low Earth orbit this problem is less than geostationary orbit or deep space but even high attitude commercial airlines flying polar routes report avionics malfunctions due to radiation events. Llis (nasa.gov). If low Earth orbit would be an interesting market, semiconductor industry may find a new market and thereby reduce the cost of chipsets that are more resilient to radiation or the material system may develop housing for computation containers that shield off radiation to acceptable levels for current chipsets.
Telecommunications
The electronics system must communicate with the hosting system by means of a telecommunications system that meets the performance requirements for a seamless uninterrupted simulated reality experience.
Existing network infrastructure will have to be upgraded to support the download and upload speeds required, but if we assume the simulation is streamed from servers in the cloud, the improvements in resolution and data are likely within the development roadmaps of fixed and wireless networks. Calculation needed. In remote, rural places, sattelites in orbit will provide high bandwidth connectivity. Today StarLink already offers fast rural internet connection through satellites in orbit, 60 times closer than normal satellites to achieve the low latency for gaming https://www.starlink.com/
How the telecommunications will be implemented is depended on the location of the hosting system. If the hosting system is located in low Earth orbit, the fixed and wireless networks on Earth need to interface through high bandwidth upload and download stations to the hosting system or connect to telecommunications sattelites that relay the data directly to the hosting system. If we submerge the hosting system near our coast lines, the fixed and wireless telecommunication infrastructure will needs to connect via under water cables to the hosting system. For the telecommunications system a submerged hosting system will likely be easier to interface with
A different approach to telecommunications would be to decentralize computing as close as possible to the user and make use of the many powerful and often inactive devices in the homes of consumers. The electronics system itself could be turned into a decentralized peer-to-peer wireless communication network. By deploying a simple device in your home or office, Helium – Introducing The People’s Network already enables people to become a telecommunications service provider themselves, earning a cryptocurrency (HNT) for the traffic through their internet devices. Thousands of existing sensors, devices and gateways can be configured to run Helium’s open-source LongFi – a combination of LoRaWan for low power wireless peer-to-peer communication and blockchain for earning credits of people using the network. There are also virtual platforms like Decentraland, The Sandbox, Cryptovoxels and many more that aspire to build a decentralized metaverse built on blockchain and peer-to-peer technology.
Energy
The simulated reality will need vast amounts of energy to perform the computations required to update the multiverse state and render the universes on the edge via fast high bandwith connections to the electronics system in close to us. Joshu Aslan performed an in-depth study about the climate change implications of gaming in his PhD dissertation, a summary can be found in this article by Eurogamer. https://www.eurogamer.net/articles/2021-10-13-gaming-downloads-climate-crisis
Already data centers use 200 TWh of electricity per year [https://www.nature.com/articles/d41586-018-06610-y]. This is just 1% of the global electricity demand. Data centers contribute to 0.3% of overall carbon emissions. The total ICT eco-system as a whole which includes consumer electronics, telecommunications networks and computing data centers, accounts for 2% of global carbon emissions. Anders Andrae of Huawei Technologies Sweden predicts that data center electricity is likely to increase about 15-fold by 2030, to 8% of project global demand [Challenges | Free Full-Text | On Global Electricity Usage of Communication Technology: Trends to 2030 (mdpi.com) ].
The energy footprint for a simulated reality is hard to estimate so we describe here a number of factors that will determine the energy demand.
First is the amount of computing devices; the higher the number of computing devices, the more energy will be required. In a simulated reality more people will have computing devices in their homes and this will also lead to an increase in computation in the cloud.
Second is the type of computing devices, smartphones and wearable devices typically require less energy than larger devices such as personal computers and cloud servers. We can expect that computation will shift into the cloud so we will see less larger devices in our personal space.
Third, is how efficient we use these computing resources. In a report by Lawrence Berkeley National Laboratory in 2016, it was estimated that if 80% of servers in small US data centers were moved to hyperscale facilities which are optimized for data center use this would result in a 25% drop in energy use. We can expect that in a society with an advanced simulated reality hyperscale centers will take up the vast majority.
Fourth, is the types of algorithms we run on the computing cores. Machine learning algorithms that are used to train applications on data sets to predict the result based on an input received are notoriously energy hungry today, see AI in the 2020s Must Get Greener—and Here’s How – IEEE Spectrum. However once a machine learning model has been trained and published, using it is relatively cheap.
Finally, there is the actual hardware on which we run the computation. Advances in semiconductor materials and chipsets design may lead to significant savings in energy. Both hardware design and software algorithms will be highly tuned to the simulated reality.
Overall, it seems logical to assume that the total ICT eco-system electricity demand (electronics system, telecommunications system and hosting system) will continue to grow in the decades after 2030, but there are two important remarks to made here.
First is that it matters how we generate that electricity. Every second 174 Pettawat (https://en.m.wikipedia.org/wiki/Solar_energy) of solar energy reaches the upper atmosphere. 30% is reflected back to space. This means some 125 PW is available. The problem with solar is that it depends on weather and the place on Earth. In addition it requires many m2 of space, energy per m2 is low Withouthotair.com. If we complement space-based hosting and telecommunications sattelites with solar farms we would be able to capture a few Pettawatts of energy every second. Using laser beams we could transport the solar energy that is not used in orbit to the submerged cloud and the electronics system in our homes. https://www.forbes.com/sites/arielcohen/2021/03/29/space-lasers-the-truth/. By the time we achieve simulated reality, we may see nuclear fusion to become economically viable https://interestingengineering.com/nuclear-fusion-is-no-longer-science-fiction. There is 33 g of deuterium in every ton of water and fusing just one gram of deuterium would release a gigantic amount of 100 000 kWh. There is around 1.37 to 1021 kg of water in the world oceans which cover for about 97% of all water found on Earth. There is more than enough deuterium according to withouthotair to supply every person in a ten-fold increased world population with a power of 30 000 kWh per day (that’s more than 100 times the average American consumption) for 1 millions years. http://www.withouthotair.com/c24/page_173.shtml
The second remark we need to make is that a simulated reality will be transformational. Entire industries will have disappeared and new ones created. A key feature of a simulated reality will be the dematerialization as we discussed before. Many of the products, services and even experiences that are now physical will be virtualized. The energy needed for supply commodities, manufacturing goods and transporting products, people around the world will be a fraction of what it is today. Even if the ICT eco-system would account for 60% of the global demand for electricity the total sum could still be less than it is today and be generated in a sustainable way.
Materials
The material supply system is responsible for ensuring a steady supply of materials and devices to the electronics, telecommunications and hosting systems. Materials include a raw commodities like silicon for chipsets, casing for circuitboards, glass fiber for cables and so on. Devices include the wearables and personal devices that surround us as we are immersed in the simulated reality as well as the computing servers that are part of the hosting system.
We can identify a number of factors that determine the material impact.
First is the number of computing devices, every computing device will have an exterior case and within it power supply, circuit board with integrated chipsets and cables to connect it all together. The more devices the more materials need. We expect an increase in the number of computing devices so this factor will be higher
Second is the design of these devices. Less material is needed if we could make large integrated chipset the size of 19 inch computing rack or if we stack multiple chipsets together in one device. As we saw before with hyperscale data center facilities we can expect innovations in material science, semiconducators and computing architecture that lead to more efficient use of materials.
Third is the lifetime of these devices. Every couple of years we buy a new smartphone and throw away or old one because there is a newer, better model. In a true simulated reality we can expect that Moore’s Law will have slowed down because improvements in the quality of the simulated experience are increasingly hard to detect. When we have achieved the level of photorealism that our eyes can hardly distinguish real from virtual, should we still upgrade?
Finally, is the circularity of these devices. Circular economy is aimed at eliminating waste and continual use of material resources. Reuse of the electronics, telecommunications and hosting system will ensure that we need less production and distribution. If devices cannot be reused their parts may be refurbished or remanufactured into newer devices. Those of you who upgrade their desktop PCs will know how your PC feels as new when you upgrade it with a new graphics card or more RAM memory. If a part cannot be remanufactured, the materials may be recycled so precious metals can be used to manufacture new parts. We can expect that a circular economy is built around the reuse, refurbishment and recycling of materials required to operate the simulated reality.
Finance
The financial system is composed of the products and services provided by financial institutions which includes banks, insurance companies, pension funds, stock exchange and many other companies that facilitate economic transactions. The financial institutions create instruments such as stocks and bonds, lend money to borrowers and maintain the payment systems. Any modern financial system aims to provide a payment system, give money a time value, offers products and service to reduce financial risk or compensate for risk taking, provides information to allow efficient allocation of economic resources, and creates and maintain a financial market that can be trusted.
The financial system that supports an advanced simulated reality is not really that different. As billions of people will work, play and entertain themselves in a global experience economy, they will need a payment system that can be trusted and enables secure transactions to take place (req 13.1). Bitcoin and other examples of blockchain payment systems such as Ethereum or Dogdecoin are seen as the future of online payment yet today they do not have stable exchange rates and cost enormous amounts of energy to produce or mine which leads many investors to opt out or avoid to step-in.
However blockchain technology if mined in a more effective, sustainable way could be interesting to solve a different challenge: Next to a stable exchange rate, purchased digital assets should have a degree of scarcity in order to keep their value. Here so-called Non-Fungible Tokens could offer a solution to this asset integrity question (req 13.2). NFT are basically a unit of data stored on a digital ledger, called blockchain, that certifies a digital asset to be unique and therefore not interchangeable [What are NFTs? Who is Beeple? A digital art craze explained – Los Angeles Times (latimes.com) ]. NFTs can be used to represent all kinds of media, photos, videos, but also digital in-game objects such as a magical weapon, a special car or avatar outfit. NFTs are tracked on blockchains to provide the owner with a proof of ownership. The simulated reality engine’s core systems (2.1.2) need to enable NFT and blockchain for the financial system to build upon but would technically implement secure ownership.
We can also expect financial products and services that help people to borrow or invest in virtual property and objects as we are used to in reality. Today many games may have their own in-game currency and market place that allow people to transfer money and in out the game (req 13.3). Roblox virtual universe has its own currency Robux where people can transfer money into Robux using gift cards but also convert Robux back into real money under certain conditions (13 years and older; own Roblox premium version; and minimum of 100K Robux; participate in DevEx program). Roblox however has the absolute right to manage, modify, suspend, revoke and terminate your license to use Robux without notice, refund, compensation or liability. Roblox also does not make no guarantee as to the nature, quality or value of Robux or the availability or supply thereof. [ Roblox Terms of Use – Roblox Support term of use 4.8, accessed 28-6-2021]. A true simulation market place will have much better investor protection and equal rights, in part enforced by the government system but also the education system that will have made people aware and conscious of the digital assets. We can also expect a consolidation in the number of in-game currencies and market places which will increase standardization and trust in the financial system and enable to government system to enforce cyberlaw more effectively and efficiently.
What remains to be seen however if the financial system plays an as important role in a true simulation as is the case in reality today. We know that to produce a real car demands a lot of raw materials (e.g. steel, aluminium), components (e.g. suspension system, brakes, wheels), labor effort and capital infrastructure (e.g. robots, production line automation) are needed which lead to high cost price and therefore a high price. But what happens if your virtual car is procedurally generated by some algorithm? Are we still willing to pay a premium price as consumer? The simulated economy will likely demonetize, the role of money will become less important, and this could either have a beneficial or negative result, as we will see in the next chapter.
Law and human rights
The law and human rights system ensures that the rules and regulations set out by the government system are maintained and protected and must be supported by the core and primary support systems. Cyberlaw is increasingly getting attention in the news. Ransomware is increasingly a threat to national security according to the Dutch counterterrorism office Ransomware a threat to national security, says Dutch counterterrorism office | NL Times and information security analyst roles are projected to grow by 31 percent from 2019 to 2029 according to the US Bureau of Labor Statistics Information Security Analysts : Occupational Outlook Handbook: : U.S. Bureau of Labor Statistics (bls.gov). For a true simulation that is pervasive in our everyday life, we need a robust law and human rights system to protect us as observers in a digital multiverse.
The law and human rights system must enforce at least safety, privacy, authenticity and human rights. First, personal physical safety must be upheld to ensure that users of the simulation are protected for electronics devices that can do physical harm (e.g. to eyes, body or brain). The mental safety of users must be protected to prevent psychological damage such as post trauma stress disorder (PTSD) from experiencing extreme violence or horror. Second, we need strong enforcement of privacy both for individuals and social (famility & friends) networks to provide a place of sanctuary where data is not shared and behavior is not monitored – or only for a small group of authorized people. Regarding authenticity, already today deep fakes and identity theft is a problem, in an advanced simulation we need at least one (master) universe (req 4.3) that provides the objective truth to warn us for an incoming asteriod impact, physical attacks on the core or primary support system of the simulated reality or any other external threat but also protects us from echo universes that break down our society. Finally, human rights must be monitored. Because we have no way of exactly predicting how the simulation will evolve, we need a strong universal set of rules. The Universal Declaration of Human Rights | United Nations drafted on December 10 in Paris in 1948 described for the first time the fundamental human rights to be universally protected and remains widely recognized and applied up until today. The UDHR declares for example in article 18 that “Everyone has the right to freedom of thought, conscience and religion; this right includes freedom to change his religion or belief, and freedom, either alone or in community with others and in public or private, to manifest his religion or belief in teaching, practice, worship and observance.”
When brain computer interfaces become highly advanced so they can read our thoughts, UDHR article 18 if implemented in the law system and electronics system, would protect users from e.g. malicious companies or governments who want to exploit our thoughts by for example selling our thoughts to advertisers or using our thoughts against us by authoritarian regimes.
To enable the law and human rights system to function well we don’t need more police on the street, we need technologies and services that provide transparency and traceability. In the previous section we described how blockchain could be instrumental to ensure asset integrity. Blockchain also provides transparency because it shows the transactions or events that have taken place where different observers agree on. If we could would log the the hypergraph calculations we could restore a previous state of the universe so the problem may not be too little but too much transparency and traceability provided by the core simulated reality system. A hacker or organization might gain access to the core simulated reality engine and get admin access to see all the events that happened and which rules were violated and use that knowledge for his own advantage. The government system which we will move to now plays an vital role in the balance of power.
2.2.9 Government
This Metaverse is going to be far more pervasive and powerful than anything else. If one central company gains control of this, they will become more powerful than any government and be a god on Earth. – Tim Sweeney (2016)
One the most important subsystem in the primary support system is the government system. A key characteristc of democratic government systems is the separation of power into three branches, a legislature, executive and judiciary branch, which is known as the trias politica model and the basis for most democratic governments today. We assume this trias politica model will also be the basis of a government to control the simulated reality.
The legislature branch is in control of making laws for the simulated reality. The requirements we introduced in the previous chapter can be seen as the rules to which the simulation must adhere. The legislature branch must make laws that prevent physical and mental harm, protect our privacy and right of sanctuary, restrict the concentration of power in organizations that develop the hardware or software on which the simulated reality depends and guard against algorithms that discriminate minorities or violate basic human rights as we discussed in section 2.2.8. The legislature subsystem will respond fast to (unexpected) negative side effects that happen because of changes in the simulated reality system itself.
The executive branch in general defends the country, protects the public order and has to ensure the laws as set by the legislative branch are applied and adhered to. Because much of our life will take place in the true simulation, the executive branch will need to rely on functionality in the game engine’s core systems to be automatically warned of suspicious activities or violation of rules by either humans or algorithms. Cybersecurity will grow further in importance to protect us from malware, computer viruses, and identity theft but also protect us from new potential threats we cannot even imagine today. Countries like Israel and Estland already have a digital ministry and several countries like the Netherlands are debating and discussing whether or not to implement such a digital ministry. We expect that in the future all countries will have a digital affairs department.
The judicial branch is the system of courts that adjudicates legal disputes and disagreements and interprets and applies the law in legal cases. Today AI is already being used to support decision making by judges in Singapore and China Technology and the future of the courts (fedcourt.gov.au). Keeping up to date with the digital transformation, ease of access and public trust and confidence are and remain key themes today and tomorrow.
Education
To build a resilient, inclusive, sustainable simulated reality it is crucial we develop an education system that teaches us our children how to use the simulated reality to build a better society. We can invest in technology to protect us from negative and unexpected side effects of the simulation and have a strong government system in place to avoid misuse and exploitation but the best defense is in the people who are the future observers and participants in the true simulation.
In chapter 3 we saw how teachers use the simulation for young children to experience history, art, other cultures and even biology, chemistry and physics in ways we cannot even imagine. We will understand empathy, communication much better when we can change our self and look at the situation from someone else’s perspective or try out different scenarios to experience the impact of choices.
A large analysis of 440 studies of K-12 students (around age 12) has shown that learning computer programming demostrably improves student creativity, mathematical skills, cognition, spatial and reasoning skills. CS helps students outperform in school, college, and workplace | by Code.org | Medium. We expect that more schools will offer computer science in elementary and secondary school in the future so kids can not only experience the simulated reality to learn new skills and knowledge but also design, program and debug their own self-made interactive, multi-sensorial experiences.
Education in colleges and universities will build on computer science, game design, and other skills and knowledge acquired but also conduct research on the risks and mitigations of the simulated reality system and the primary supporting systems. This research will be performed partly outside the simulation but the simulation itself will also be used to recreate alternative universes to test out hypothesis and witness behavior that is simply too complex to understand. The technology and experiences we design are not neutral. New products and process we bring into the world determine the way we see it, just as the way we see the world determines how we design new products. Ethical reflection through design – Research | DDW
The secondary support system consists of all the subsystems that do not directly support or enable the simulated reality system (and are not part of the simulated reality itself). The secondary support system includes for example agriculture, mining, manufacturing, logistics, healthcare, hospitality, retail, leisure and much more. We will not describe this secondary system in the same way as we did for the simulated reality and primary support system but return to key changes in the secondary support system in the next chapter when we discuss the benefits and risks of a true simulation.
