Midori will have provisions for distributed concurrency-or cloud computing-where application components exist in data centers.
Midori is an offshoot of Microsoft Research’s Singularity operating system, the tools and libraries of which are completely managed code. Midori is designed to run directly on native hardware (x86, x64 and ARM), be hosted on the Windows Hyper-V hypervisor, or even be hosted by a Windows process.
According to published reports, Eric Rudder,
senior vice president for technical strategy at Microsoft and an alumnus of Bill Gates' technical staff, is heading up the effort. Rudder served as senior vice president of Microsoft’s Servers and Tools group until 2005. A Microsoft spokesperson refused comment.
“That sounds possible—I’ve heard rumors to the effect that he [Rudder] had an OS project in place,” said Rob Helm, director of research at Directions on Microsoft. He noted that it is quite possible that the project is just exploratory, but conceivably a step above what Microsoft Research does.
Source SDA Times.
The secret project from Microsoft attempts to seek a future in a world of freeware, open source and cloud computing. The Internet craze is baffling.
One of Microsoft’s goals is to provide options for Midori applications to co-exist with and interoperate with existing Windows applications, as well as to provide a migration path.Building Midori from the ground up to be connected underscores how much computing has changed since Microsoft’s engineers first designed Windows; there was no Internet as we understand it today, the PC was the user’s sole device and concurrency was a research topic.Today, users move across multiple devices, consume and share resources remotely, and the applications that they use are a composite of local and remote components and services. To that end, Midori will focus on concurrency, both for distributed applications and local ones.According to the documentation, Midori will be built with an asynchronous-only architecture that is built for task concurrency and parallel use of local and distributed resources, with a distributed component-based and data-driven application model, and dynamic management of power and other resources.Midori’s design treats concurrency as a core principle, beyond what even the Microsoft Robotics Group is trying to accomplish, said Tandy Trower, general manager of the Microsoft Robotics Group.The Midori documents foresee applications running across a multitude of topologies, ranging from client-server and multi-tier deployments to peer-to-peer at the edge, and in the cloud data center. Those topologies form a heterogeneous mesh where capabilities can exist at separate places.In order to efficiently distribute applications across nodes, Midori will introduce a higher-level application model that abstracts the details of physical machines and processors. The model will be consistent for both the distributed and local concurrency layers, and it is internally known as Asynchronous Promise Architecture
In that scenario, operating system services, such as storage, would either be provided to the applications by the OS or be discovered across a trusted distributed environment.Likewise for local concurrency, Midori will have a programming model, a platform stack and execution techniques that are intended to help developers write applications that can safely and efficiently use a greater number of hardware threads than is currently feasible. Elements in local parallelism interact through shared memory, which is the huge difference with distributed applications, said Microsoft distinguished engineer John Manferdelli, in a separate interview.“Mere mortal developers need a programming model/application model that lets them distribute processing to massively parallel devices without having to become experts," explained Forrester Research senior analyst Jeffrey Hammond in an e-mail. “Even with the quad-core Intel chips today, you have to have specialist teams to take full advantage of them,” he added.These design goals affect aspects of the system that include its application model, scheduling and storage. Indeed, big changes are in store for Microsoft developers.Programming with MidoriThe Midori programming model will tackle state management, which Microsoft admits in its documentation is a challenge in Windows, by migrating APIs, applications and developers to a constrained model.Other objectives are eliminating dynamic loading and in-process extensions; developing a failure model based on reliable transactions, so the system understands exactly which processes are impacted by a cascading failure and how to restart the computation; and having a standard way of dealing with latency, asynchronous behavior and cancellation, throughout the stack.Forrester’s Hammond said that doing away with dynamic loading and in-process extensions was worrisome. “I’m going to assume that eliminating dynamic loading doesn’t prevent dynamic language execution,” in virtualized interpreters. Microsoft, he added, must “be clear that restricting dynamism at the OS level will not impact dynamism at the programming level.”The Midori programming model will be particularly useful for service-oriented architectures, by allowing for the decomposition of applications into services that can be partitioned across tiers.Hammond said that having SOA go into the runtime makes sense, as that would remove a certain amount of middleware complexity. “Why shouldn’t the average developer begin to think in terms of lightweight, asynchronous services?” he asked. “After all, that’s the migration path we’re seeing on the Web.”In a possible link to Microsoft’s
Oslo composite application initiative, the programming model will have a dependence on metadata, with the aim of allowing the system to more reliably manage applications.“This allows existing development tools to be easily repurposed while a lot of the complexity is hidden from the developer that is using it. We essentially see declarative programming replacing imperative programming at the OS level,” said Hammond. He noted that by having Oslo in place first, Microsoft would have an easier time when it begins the migration from today’s Windows applications to Midori or hybrid applications.“I wonder if [Microsoft] concluded this sort of 10-year sea change was needed before kicking Oslo into high gear?” asked Hammond.The Midori documents indicate that the proposed OS would have a non-blocking object-oriented framework API. This would have strong notions of immutability—in the sense of objects that cannot be modified once created—and strive to foster application correctness through deep verifiability by using .NET programming languages. At the presentation layer, Microsoft is making a clean break from the existing Windows GUI model, where applications must update their display on one and only one thread at a time, and the associated problems that affect OS stability and make it more difficult to write multithreaded applications.The Midori documents indicate that the company has not decided what user interface abstractions are appropriate when applications cut across boundaries, or how to combine the best qualities of rich client applications and Web applications.“A lot of these problems are being solved, at least partially, by the ideas of store-and-forward and message synchronization,” Hammond noted. “Google Gears, Adobe AIR, even the mobile OSes with things like SMS can handle occasional connectivity. Why shouldn’t this be built into core OS communication protocols, especially if they are asynchronous by default?” he asked.Midori’s applications would be created using .NET languages that will be compiled to native code using the
Bartok compiler and runtime system, which is presently a Microsoft Search project. The Bartok compiler can typecheck machine code programs for programming errors thanks to its use of an intermediate typed language, according to the company.Microsoft’s objective is to force developers to create applications that are correct by construction, and it has repeatedly pledged to shore up the overall security of the operating system. The use of .NET languages in Midori will create a new, safer programming model with higher-level reasoning, predicted Larry O’Brien, an independent analyst and consultant who writes the Windows & .NET Watch column for SD Times.Another advantage of using .NET languages is retargeting, O’Brien said. “A very smart compiler or runtime could move a calculation onto a GPU or distribute it across cores,” he explained.However, O’Brien observed that some of the onus for making this work might end up on developers. The Midori documents note, somewhat ambiguously, that applications were expected to “contain sufficient latent parallelism.” Reacting to that, O’Brien asked, “In a world where Moore's Law doesn't imply the speeding of individual components, where does this expectation come from and who holds it?” The Midori design will also incorporate a type-safe abstraction set based upon a .NET language, say the documents, in order to provide a system binary interface that will eliminate the current break between the operating system and virtual machine runtime.The abstraction set will eliminate an entire class of programming errors that stem from bad pointer arithmetic, enable the changing of the boundaries between privileged and unprivileged code, and provide for universal application analysis and instrumentation, Microsoft reasons.The use of an abstraction set, said Hammond, “reflects the reality of programming today: The vast majority of professional developers, especially those in IT and out on the Web, don’t deal with low level constructs. Unless you’re a game developer, ISV or systems programmer, there really isn’t the need to do pointer math.” Hammond believes that it would be advantageous for Microsoft to create a programming model that “mere mortals” could actually understand, akin to the early days of Win32 when Visual Basic was born.Even though memory safety and type safety are deeply integrated into Midori’s design, Microsoft has yet to determine just how low to permit the Bartok runtime to delve into the kernel, or alternatively, whether it will allow some unmanaged processes to rely on Midori’s hardware address spaces.The company also acknowledges that thread safety remains elusive, and it is investigating transactional memory as a proposed solution. O’Brien noted that there is significant indecision in the program model. “On the one hand, the phrase ‘strong notions of immutability’ has serious implications if meant formally, but elsewhere we see ‘thread-safety remains elusive’ and a laundry list of things that might contribute to a solution,” he said.Backwards compatibility with legacy applications and hardware has also been considered; several Midori components already run on Windows as well.The fundamentalsUnlike Windows, Microsoft intends for Midori to be componentized from the beginning to achieve performance and security benefits. It will have strong isolation boundaries and enforced contracts between components, to ensure that servicing one component will not cause others to fail, while keeping overhead minimal.At its lowest level, Midori has two separate kernel layers: a microkernel comprised of unmanaged code that controls hardware and environment abstracts, and higher-level managed kernel services that provide the full set of operating system functionality.The OS will have a single scheduling framework for all device types, known internally as the Resource Management Infrastructure (RMI). RMI will have provisions for resource accounting, quotas and management; resources including IO bandwidth, memory, power and response time will be monitored.
Microsoft believes that power-based scheduling will be particularly useful for mobile devices. It is considering creating a layered, thin platform for such devices, but it remains unclear how far the company can go with a single code base.The ecosystem of devices is a major consideration in how Microsoft may choose to implement storage, perhaps by teasing functionality out of the OS and moving it into distributed services, with parts of the service being executed on the device itself.“In this scenario, you establish Midori not so much as a replacement for Windows,” Hammond noted, “but as the hub of a new type of distributed system which Windows machines connect into until they are no longer needed,” in a fashion similar to IBM’s multi-year transition path for moving its iSeries customers to pSeries and xSeries platforms.Hammond went on to forecast that there will be a deluge of mobile devices introduced over the next several years built with similar hardware, but with a range of different power and form factors.Microsoft also envisions higher-level opportunities for storage, including compliance, compression, consistent replication, computation close to data, encryption, indexing and search, as well as storage in the cloud. Midori provides a built-in multi-master replication for complex data.Scheduling, a process that allows multiple processes to run on the processor at the same time, will be integrated in Midori at the user-mode application level, from both the desktop and across distributed applications in the cloud. Its distributing scheduling may include active task migration, an activity that today is performed by hypervisors.Notably, Midori’s scheduling may provide hooks for third parties to integrate software that asynchronously updates scheduling tables.The intention is to enable developers to create collaborative Web-like applications, such as active documents, that operate safely and securely at the OS level. Resource quotas will be used to prevent denial-of-service attacks.“This is the second attempt at re-implementing OS scheduling that I’ve seen announced in as many months,” Hammond remarked. “[Steve] Jobs talked [at the Apple Worldwide Developers Conference on June 9] about how Snow Leopard was going to have a new scheduling framework that would make take advantage of multicore easier for OS X developers. This seems to reach similar conclusions, and then take it to the next step in terms of scheduling flexibility,” he added.No timeframe for development has been set for Midori, which Microsoft technical fellow Burton Smith says is a research project. A spokesperson added that Midori is one of many incubation projects across Microsoft Research.
Posts
