Observations from the field
Insights into technology, business, process, science, maths and life in general
-
Is Browser an OS?
Posted on April 17th, 2009 No commentsNowadays we do most of our work within the browser than outside it. So a logical question would be - what is the need for any applications outside of the browser? When I can write, edit and save documents with Zoho or Google why do I need a seperate MSWord App? If sending emails through Web Outlook is as easy as sending it through desktop app why do I need a desktop version? Valid questions and I would say may be in future there would be no need for these desktop apps.
Then you go further and suggest that you anyway store all your data in google or skydrive and hence what is the need for a big harddisk on my comp? Fair enough and I would think in future when the world will be extremely connected (wired and wireless) there would be no need for a large local harddisk. Then you go further and say if there are no apps apart from browser and there is no need for any large local storage, then why do we need to think of OS and Browser as separate entities? It seems as if OS’s sole purpose would be reduced to running the browser effectively. The processor would be dedicated to doing just that. May be we just need an OS supporting a standards compliant effective browser which can just run javascript (not even plugins like flash and silverlight).
Who then gives those javascript? The apps you are runnin inside the browser.But then where are these apps running? In the browser and in the cloud using someone else’s processor cycles (may be Google) using someone else’s storage (May be Microsoft). So if you look at at current day webapps they would typically be rich UI mashups where sections in single page will be independently interacting with different service end points. Infact it is quite possible that in a single web page in the browser javascripts code elements from multiple cloud apps from different cloud endpoints could be running. Here we have an unique problem where one cloud app could get access to another cloud app using its javascript. Here we are in a sense talking inter process communication but without much safeguards wrt to security. There needs to be some kind of sandbox within which javascripts of a particular cloup app should function in. This is similar to domains inside a .NET process. This is a familiar problem much before cloud computing and is called cross-site scripting vulnerability. But this vulnerability attains great significance in the cloud computing age.
This issue have been analysed by Microsoft in its research division and they have come up with a proof of concept browser called Gazelle. I think we would be seeing an overhauling of browser standards and implementations in the changed cloud computing age.
-
Workaround for IPv6 Default Address Selection Implementation in Vista
Posted on April 1st, 2009 1 commentMicrosoft implemented the standard (RFC 3484) specifically section 6 rule 9 in Windows Vista and Windows Server 2008. Microsoft I think is the first major client OS to do so.
Rule 9: Use longest matching prefix.
When DA and DB belong to the same address family (both are IPv6 or
both are IPv4): If CommonPrefixLen(DA, Source(DA)) >
CommonPrefixLen(DB, Source(DB)), then prefer DA. Similarly, if
CommonPrefixLen(DA, Source(DA)) < CommonPrefixLen(DB, Source(DB)),
then prefer DB.This rule may have been introduced to route the request to the closest location rather than choosing the destination address randomly when presented with multiple A records. But in IPv4 it doesnt work that well as internet ip addresses doesnt correlate fully with geographical location. More than that if your behind a NAT gateway your IP address the DNS client in your Windows PC sees is one of these “10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16″ which doesnt provide much information on your geographical location and hence cannot be used to gauge location proximity resulting in wrong routing decisions if you implement Rule 9. This is not just theory but has happened in practice as explained by Dr Plotka in his blog and hence is extremely problematic. So in effect Microsoft kind of broke the internet traffic by correctly implementing the standard.
But the standard does provide an optionality to this implementation as follows:
Rules 9 and 10 may be superseded if the implementation has other
means of sorting destination addresses. For example, if the
implementation somehow knows which destination addresses will result
in the “best” communications performance.Infact in the case of IPv4 communication I would say you would be better of selecting randomly than following the rule 9 suggested by the standard. All along all DNS clients in Windows PC chose randomly by default. This changed only in Windows Vista and Windows Server 2008.
Realizing the problem Microsoft has caused they have published a workaround to revert back to the randomn selection behaviour.
Please read the Microsoft support article here.
-
Guardian is now uttering successive chirping sounds!
Posted on April 1st, 2009 No commentsYes! Guardian has decided to twitter their news! They believe that any news can be told in 140 characters (maximum limit for twitter messages). I already subscribe to Sciam and Onion and they do keep me quite busy I should say. But then they do not really convey the whole story in 140 characters but just the first part of the first line followed by … and tinyurl.
Reuters doesnt do that but provides the gist of the whole news story in the tweet (one twitter message) and then follows it up with an url. Afterall Reuters were the original news twitterers (if such a word exists) as they twittered much before twitter.com came into existence. Infact Reuters’ business model , in their early telegram age, centered around quick transmission of important tweets. Infact in places which werent covered by telegraph lines they even used birds (pigeons) to carry their tweets!.
-
OS Provider Hosted Mobile App Stores - Is this a temporary phenomena?
Posted on March 31st, 2009 No commentsThere has been great frenzy in launching mobile app stores by different Mobile players after Apple launched its app store. The latest to join the bandwagon is Microsoft which is planning to demonstrate their version of app store - “Windows Marketplace For Mobile” at CTIA 2009. This will come as an inbuilt app in Windows Mobile 6.5 which is expected to hit the market in the last quarter of 2009.
Apart from Apple and Microsoft, we have other players in the mobile space who have opened their own markets.
- Google - Android Market
- Nokia - Ovi Store
- Blackberry - App World
- Palm - Software Store
Let us briefly look at the desktop market.
The top desktop operating systems we have are (from wikipedia):
- Windows (XP,Vista..) ~ 90.5%
- MAC OS X ~ 5.54%
- Linux ~ 1.13
Now the obvious question we want to ask is - do these Desktop OS’s have a Desktop App Store packaged along with the OS? None have it. This app store concept is a new concept confined to just Mobile OS. Such an idea was brought into market initially by Apple for iPod and iPhone. Following the success of this business model, this was later copied by other players.
In the same way should we expect a App Store for desktop apps from Apple? Is it a feasible model for the three main players in the desktop market?
There is something called store.apple.com which sells third party software which run on Mac OS which is similar to the App Store for iPhone but accessible over the internet. Atleast Apple has a place to consolidate and sell stuff which work on Mac OS while Microsoft has no such store. On the other hand Microsoft have the Windows Market Place which certifies if a particular software is compatible with a Windows OS, lists them in their product list and provides a link which takes you to the software publisher’s site. Yet these sites dont cover all the apps available in Windows OS or Mac OS. For the desktop market typically the internet, retail chains, ecommerce shops, eBay like market places act as the gateway to buy and download desktop software.
With the consolidation and maturity of the Mobile Market, the number of apps out there would become humungus that it would soon be infeasible for the Mobile OS players to go through an elaborate approval process which currently they are doing and list all those. Already iPhone third party app developer community is feeling the pain of the slowness of the Apple’s approval process and also they face the possibility of rejection. Infact to circumvent this approval bottleneck an app named Cydia has already cropped up which is mimicking the iTunes Appstore and has started acting as an alternative to iTunes in iPhone to source iPhone apps from a wide variety of sources.
With the increase in the amount of power and capability packed into Mobile OS, the complexity and the size of the apps in Mobile OS will start equalling the apps in desktops. The browser experience would also become enriched and as friendly as the experience in normal desktops. With all these developments, along with market consolidation in terms of players and increase in number of applications developed for Mobile OS, it is possible that the application distribution model will start moving towards the distribution model used for third party desktop software. In that scenario the current OS provider hosted Appstore will slowly fade away. May be it too early to call - let us see how this market matures.
-
Algebra and Geometry
Posted on July 19th, 2007 No commentsAlgebra and Geometry
It is always interesting to note in an enterprise software development team, there would be some specific people who can write better SQL Queries than others. There are some who are good at OO concepts, complex algorithms but loathe writing SQL Queries. I would compare normal code to Algebra and SQL to say Trignometry or Geometry. These are two very different problem spaces in mathematics. But if you could take the spatial element of geometry and transform it through the aid of what is called coordinate geometry, you could represent circle and lines using algebraic equations. Infact mathematicians sometimes use geometry to solve algebraic problems as in the proof of Fermat’s theorem by Andrew Wiles. The kind of geometry Andrew Wiles used in his proof is called Algebraic geometry.Like how initially geometry and algebra formed separate problem spaces, writing stored procedures and writing other kinds of enterprise code (Business Logic and Presentation Layer) are regarded as distinctly different problem spaces. But in this case some would argue that they truly are different unlike the difference between geometry and algebra. Let us investigate if it is true.
Set theory and SQL
At the very core SQL Query’s job is selection of a subset of elements from a bigger set. When I want to select all the customers who belong to California , I am interested in the subset of the “customers” set. [Please note that all the rows (elements) in the “Customers” table in your database constitute yet another subset of the set “all the people in the world”. By adding a new customer row entry you are increasing that subset and by removing an entry you are decreasing that subset.]The “where” clause in a SQL Query specifies the selection criteria, described in terms of the properties which define each element of the subset. When we have two conditions in the where clause, then that is equivalent to extraction of two subsets for each condition, and making intersection/union of these two subsets based on the conjunction (OR, AND…).
The “Join” clause is another interesting keyword, which correlates two totally different sets like say “Customers” and “States”, but has a relationship “Customers reside in a State”. Here we take two sets and do a “Set Product”. A Set Product is a set whose elements are pairs consisting of one element from each operand set. So if there are 10 students and 2 courses, the number of elements in the set product will consist of 20 elements tying each customer with all the states. But then if we impose a constraint that a customer resides in only one state (by making say state a column in customer table – popularly called foreign key relation), then we could create a subset from the “Set Product”, using this constraint and we realize a Join. Again Join is a process of extracting a subset after doing a product operation. Similarly all other SQL clauses could be explained through Set Theory and that would be left as an exercise to the reader!
Declarative vs Imperative Programming
Having covered in brief the underlying set theory aspect of SQL, I would now like to concentrate on another interesting aspect of SQL. People differentiate between declarative programming and imperative programming. SQL is a pretty high level declarative language compared to say a language like C#. But then C# could be called declarative compared to say assembly code. But in general, declarative coding indicates that the coder provides less detail on how to do something rather specifies only what to do. It is up to the instructed (SQL Engine or the compiler as the case may be) to work out the mechanics of how to do. More you specify “how to do”, more imperative your code becomes. Now using code, I would say you could MicroManage till you reach “Processor Instructions”, but if you want to control more and don’t feel that the processor is not doing a good job at executing your instructions, then you might have to think of writing your own processor. But whatever way we think, definitely C# coding seems to be more imperative than SQL queries.But there are places where C# code has a tinge of declarative programming, for instance in the use of attributes. Attributes are used to decorate a class or member method, indicating to the runtime as to what additional functionality we would like runtime to provide the class, but stopping at that and not elaborating as to how to achieve it. It is left upto the runtime or provider classes which hook into the runtime to make sense of such attributes and act accordingly.
Lists and Iterators
We have all worked with lists and have iterated through the lists. Lists of objects can be thought of as sets of elements. All the set operations which we explained above done using SQL can be done on lists too through explicit functions operating on the sets. List after all is a data structure quite similar to a table, with columns representing properties and foreign keys representing reference properties. Say we need to use C# to do the same stuff we have done using SQL in extracting relevant subsets from a List, how would you go about it.Using the same example, suppose we want to select from a Customer List, whose entries which correspond to the customers who are from California , what would we do? We would typically iterate through the list and investigate each customer element in the list and find out if the State property is equal to California and if so we would add to a result list. In the end we would return this result list which is the subset of original list. This is similar to what SQL Server would be doing behind scenes when we execute a SQL Query though here the source list is an in-memory list while there the list resides in the database.
All kinds of lists, needs to be iterated to traverse the list elements. A List which implements IEnumerable interface or a generic list which implements IEnumerable interface, which in turn has a function GetEnumerator which inturn returns IEnumerator reference, then such a list is called a “Sequence”. In these list to iterate we could use the “for each” statement. IEnumerator has three methods to traverse the list, MoveNext, Current and Reset and the “for each” internally uses these methods. IEnumerator allows Forward Only traversal. Such well defined iteration interface mechanism is very useful for external functions and classes to act on Lists without ambiguity and uniformly.We have class and class member functions. Let us say, we have written a class, but we want to add one more function to the class, but say I don’t or cannot change the code of the class, what do we do? This is also true if you have a class written by a third party provider, but you would like to add an additional method to the existing set of member methods. Typically this is performed through by composition-delegation or implementation inheritance, adding the additional functionality in the composing or derived class. Instead of going through the headache of doing all that, C# has provided what they call syntactic sugar, to inject an extra method into the existing class definition, and can be called with the same syntax used in calling existing class methods. But please note that though we seem to inject a member method, this member method can only access public members of the class! So though they seem to give a feel as if they are a part of the class, they are in essence outsiders. These members are defined as static members of a static class with the first parameter as this
.It is as shown below:namespace Extend
{
public static class ExtendUtil
{
public static string ExtensionMethod(this OriginalClass oc)
{
-
}
}
}
The member method “ExtensionMethod” of the static class ExtendUtil extends the OriginalClass with this new method.
It can be invoked as follows.
OriginalClass oc = new OriginalClass();
Console.WriteLine(“oc.ExtensionMethod = {0}”, oc.ExtensionMethod());
Please note that the call will be translated to in IL as follows.
call string Extend. ExtendUtil::ExtensionMethod (class OriginalClass)
This clearly indicates that this is just Syntactic Sugar as ultimately this call is getting converted to an external call and also the extension method has only access to public members of the original class reiterating the same point.
Lambda Expressions and Anonymous Delegates
delegate R Func(A arg);
The above declaration is a declaration of a generic delegate type Func which takes an argument of any type A and returns a value of any type R.
A specific instantiation of that would be like below.
Func<int, int> f1 = new Func<int, int> (MethodClass.OnlyMethod);
But then in certain cases you do not want to write a class and write a method, when you are just interested in a single function. Infact if you look at it we are more and more moving towards the creation of a global function. In order to remove the trouble of writing MethodClass and have a member OnlyMethod, C# gives a nice way of creating a function inline through what is called anonymous delegates. It can be done as follows.
Func<int, int> f1 = delegate(int i) { return 2*i; };
Now we could call this function like below.
Console.WriteLine(f1(12));
What we have achieved is creation of a nominal global function without having an explicit class. If suppose the function is a very straight forward algebraic transformation like the case above, then we could simplify it further by writing it as below.
Func<int, int> f1 = x => 2*x;
The expression “x => 2*x” is called a lambda expression which takes input parameter as x and returns out as 2*x. So if you look at it Lambda expression can be achieved through existing c# language constructs and is not a fundamental keyword, but as they call it is just syntactic sugar. Here the Lambda expression is just an anonymous delegate which cannot be inspected and changed at runtime. If at all you need to inspect it, you would need to inspect IL. Expression trees are a higher form of Lamba expressions where the expression or algebraic computation is stored as Data, which can inspected and changed at runtime.
Putting it all together - Realizing Algebraic Geometry
Now we have looked at many different concepts and are ready to approach LinQ in its elementary form. LinQ expands as “Language Integrated Query”. LinQ as we see sounds as if it is trying to close the gap between the imperative C# and declarative SQL. Let us see how it does that taking a simple case.
Suppose I have a Customer table and I would like to get the list of customers who belong to a particular state. Using ADO.NET you could execute a SProc (using say the data access application block) passing the state as a parameter. Inside the stored proc it would use that parameter in the where clause – Select * from Customer where state = @state.
Suppose I want to achieve this on a customer list using C#. This may be how I would go about it. Let us say I have a customer list and I have method to populate the list and I populate the list like below:
List<Customer> customers = GetCustomerList();
Then I would need now to get the list of customers belong to a “ California ”. That is easy.
Write another function
List<Customer> FilterCustomerListByState(List<Customer> customers, string state)
– which takes the state as a parameter. Inside the function we can iterate the list using the provided enumerator and create a new list of only those customers who belong to that state. But what happens when I now decide to want to get customers who do not belong to California ? Again I have to write another function for the same. Instead let me write a generic filter function, which accepts a filter criteria and uses the enumerator to iterate the list, apply the filter criteria and returns a new list with elements only satisfying the filter criteria. How will that function look??
List<Customer> FilterCustomerList (List<Customer> customers, filter)
But what is the Type of filter? If it is a string, then it should be in a defined format and we need to parse it. That might not be the best way to do it as we need to define an elaborate filter language and grammar. How about saying that filter should be a function which tells whether a customer qualifies or not? This is normally called a predicate.
In our case it could be a delegate of the following type.
public delegate bool filterDelegate (Customer cust);
We can have the real function implementation in a class CustomerPredicatesImpl and let us say the function is named fiterImpl.
filterDelegate filter = new filterDelegate(CustomerPredicatesImpl.fiterImpl);
After we get the delegate instance, we could pass it to the FilterCustomerList function like as below.
List<Customer> filteredCustomers = FilterCustomerList(customers, filter);
If we write the function FilterCustomerList as a Extension method for the List class like how we discussed before then we could have written the invocation as follows, which is more friendlier
List<Customer> filteredCustomers = customers.FilterCustomerList (filter);
Suppose I where to use anonymous delegates and Lambda expressions, then I wouldn’t need to do that roundabout way of creating the filter delegate and I could have written like below.
List<Customer> filteredCustomers = customers.FilterCustomerList (c => c.State == ” California “);
Here the lambda expression c => c.State == ” California ” would be taken and converted to an anonymous delegate which fits the parameter type of method FilterCustomerList.
If we generalize the extension method not extending just List, but any class which implements IEnumerable then that is what is called as the “where” query operator in LinQ which is defined as below.
public static IEnumerable Where(
this IEnumerable source,
Func< SPAN>bool> predicate);This is nothing but the extension method which we were discussing above.
So the statement above could be written as:
IEnumerable filteredCustomers = customers.Where (c => c.State == ” California “);
and suddenly it starts looking like LinQ.
There is a select operator which is also an extension method similar to Where, which projects or transforms the input set to an output set of a different type or same type.
Suppose say we want to only select customer names, and not the customers themselves, then it could be done as follows.
IEnumerable<string> filteredCustomers =
customers.
Where (c => c.State == ” California “).
Select (c=> c.Name);
Another way to write this is as below by removing some brackets and periods which can be inferred by the compiler.
IEnumerable<string> filteredCustomers = from c in customers
where c.State=” California ”
select c.Name;
Now doesn’t this look like a SQL Statement? But you are staring at C# code though!!
-
Vista I/O - Whats New??
Posted on July 19th, 2007 No commentsI was listening to a webcast in Channel 9 on Windows Vista I/ONarayanan Ganapathy amazed me when he said there are priorities for I/O in Windows Vista, like we have thread priorities. I hope all the indexing services listen to the webcast and run under idle priority.One more thing which was really cool was the concept of “Bandwidth Requests” where you could specify when making a I/O request to the I/O manager to satisfy the I/O request at a particular I/O rate. These are obviously hints which OS needs to use when it does its I/O operations. The OS maintains seperate prioritized I/O queues based on the hints provided and satisfies the requests at the requested I/O rate (or SLA - System Level Agreement
The last aspect which was worth noting is the ability to cancel a synchronous I/O operation which is not possible to do in XP and causes most of the hang situations. This is good as not many use Overlapped I/O and prefer to use Synchronous I/O as it is easy to program.
It is really a nice watch, and I strongly recommend the same..
http://channel9.msdn.com/Showpost.aspx?postid=239845
-
Agile! Agile! – Where Art Thou - Part 2 (Agile Workplaces)
Posted on July 19th, 2007 No commentsI said in my last blog post…“In the next part in this Agile series, I will delve more into what constitutes those SCRUM techniques and methods which create the controlled chaos atmosphere.”“Atmosphere” is one of the key words here. We will be dwelling on one part of that in this post. One of the basic tenets of Agile Dev is excessive interaction. Agile extremists even sometimes encourage over communication. The idea is that such communication would easily help in removing disconnects and enriching solutions with varied ideas. The Agile methodology encourages a lot of face-face communications more than verbal or document based communication. The cool thing in face-face communication is that it enables you to communicate a lot in a shorter span of time. Besides that, more importantly, it also allows you to deviate and explore other unconnected but very relevant areas which wouldn’t have been discussed in a formal structured mode of communication.
Now let us look at how these needs affects workplace construction. If the amount of time you spend on face-face communication is considerable and the number of times you have such communications significant, then it makes sense to shrink the physical separation between two people. This would mean, having a separate room for each person doesn’t make much sense at all.
Some of the issues isolated rooms bring about:
- The time taken to go from one room to another room to have the discussion is high and hence sometimes people may avoid such discussions
- The general reticence of individuals to not disturb the feeling of privacy, an isolated room creates
- An isolated room makes a person psychologically feel that ‘he’ his are trying to do ‘his’ job and ‘earn a living’, rather than feel that he is part of a team which is assembled for the purpose of completing a particular project.
- Isolated rooms tend to make individuals form roots to the room they are in (Photographs, nice vases…) and would bring to their tiny office a feeling of a second home. Though this may be good in a way, it tends to make the individual distracted from the fact that he is in this building because he has a task at hand to do. The thought process should be more in the lines of “There is a project to do and I am working for it” rather than “I am there and I am working a project”. I hope you understand the subtle difference. In the former case, when the project vanishes, you become a nomad again searching for the next project. If there are no isolated rooms, but only project rooms, you truly become a nomad when you are not in a project. There the work place attains its true significance. You come to work place because there is some work to do, but not because there is a workplace to go to. The place where you go like that is only your ‘Home’. Workplace is not ‘Home’.
- One slowly gets intolerant to other intrusions, or too private to open out to others. Both are blockades to free communication which will show in work.
- For discussions to happen conference rooms have to be booked before hand and its availability is time-bound. Hence discussions have to be hurried, takes time to be planned, thus resulting in postponing and discounting of critical communication.
So how do we construct our workplace to suit SCRUM or Agile development? This is a very interesting question. Microsoft’s patterns and practices group which follow the SCRUM model has attempted to answer that in practice.
Please have a look at the below webcast to see how they have managed to model their workplace to suit the Agile model of development.
http://channel9.msdn.com/showpost.aspx?postid=238321
-
Fundamentals of Cellular Communications - 1
Posted on July 19th, 2007 No commentsThere are two popular mechanisms which are used to transmit data/voice through available frequency spectrum. They are CDMA and GSM. I will try to explain these technologies using analogies. Initially we had something called FDMA (FDMA - I think expands to Frequency Division Multiple Access). We need to start with that to explain the subsequent successors.The concept is pretty straight forward. On a Friday night say you decide to go bowling along with some of your friend. You reach a bowling place and you see it is pretty crowded, but then you notice some alleys are free. You go to the person in charge and he locates a free alley and allots it to you and then you start bowling in that alley. There are around 20 bowling alleys. Each alley is clearly separated from the adjacent bowling alley so that your bowling ball doesn’t interfere with the balls in the other bowling alley.Similar to that, in FDMA there is an allocated frequency spectrum which can be used for mobile communications. This frequency spectrum is divided into smaller frequency bands (subdivisions), each band/channel separated from each other to avoid interference. Each frequency band is allotted to a conversation/ call.
This is similar to how your radio channels share bandwidth, but then radio channels are different in the sense, they occupy the channel for a greater area depending on the power of the transmitter. In the case of cellular communications, the frequency sharing is limited to a defined area called cell and in a different cell the same frequency band is used by a different call.
We will get to these details later. But clearly we don’t have infinite frequency spectrum width and definite allotted width. So as the allotted spectrum is limited, when demand (no of calls in a cell region) increase, the channels get used up in that region and there is no room left for more conversations, similar to the days when you went late and didn’t get a free bowling alley. The number of simultaneous conversations with in a cell is limited to the number of frequency bands.
The FCC licenses these frequency bands to specific companies who bid for them. These licenses are issued territory wise for these frequency bands. So frequency band in one state might be owned by one cellular company while in another region by another cellular company.
So the obvious question is why don’t we make the frequency bands smaller and accommodate all people or in other words why don’t we have smaller bowling alleys??!!
We will answer the question in detail in the next blog ….
-
Vista Memory Management - Super Fetch
Posted on July 19th, 2007 No commentsI think Microsoft has been doing serious thinking in the area of OS performance and they better be as they add complex features which eat up performance. This is evident in the latest super cool feature called Super Fetch which is touted to bring in a lot of performance gains.It essentially predicts which pages should be brought to memory by evaluating (through time) the user’s disk access behaviour for that computer. Typically say when you go out for a coffee, a service runs or an autoupdate program runs or a virus tool runs. In that time, these background programs based on the files it accesses replace existing pages and bring in new pages into memory. After your coffee session is over, you start working and you find the computer is slow - this is because the files which were important just before you went for your coffee are no more in memory and is lying in the disk to be fetched again and placed in the memory.Suppose if the OS restores the memory state similar to what it was before you went for a Coffee or atleast keeps the files in memory which you think you will be accessing or working on after your Coffee break, how cool it would be?
This is exactly what Super Fecth tries to do by understanding user’s presence and user disk access behaviour. It figures out what memory pages you would be interested in future though they are not currently demanded by any program and brings them to memory before hand. When the future happens magically the data page is already in memory and need not be fetched from the disk! Hence your file open is done so fast!! Ofcourse the fetching of these futiristic forecasted pages will be done during idle time with idle I/O priority.
They also have feature called Ready Boost/Drive which uses a flash drive or a USB key drive to be used as an intermediatary memory layer for the disk drive. So you could mimize disk access by inserting a flash drive which acts as intermediate non volatile storage area. As the flash drive access time will be much lower than disk access time this intermediatary storage layer is a boost for data access performance.
When you complain of performance issues, typically the sysadmin says, please run the defragmenter. Now this is very tedious task as the defragmenting a 80 GB disk is a laborious process and takes lot of time. But then is there any use in compacting file fragments of files which you never access? This is where superfetch gives clues to a mini defragmenter as to which of the files it needs to defragment. As the files of interest would be a very small subset compared to all the files in the disk, the defragmentation process takes very little time. As we would only need to access these predicted files of interest during our day to day operation and that set is defragmented regularly and quickly, the overall disk movement is reduced in fetching relevant files, thus increasing the disk access performance to a great extent.
Till now we were fetching and placing in memory more data than what is demanded by forecasting demands. But let us look at what happens when computer hibernates. The whole memory state needs to be captured in the disk and restored when we wake up from hibernation. Suppose we had a 2GB RAM, that would be a lot of Disk Access to store everything into Disk and here Vista does an intelligent prediction of what subset we need when we come back and only stores those memory pages in the disk. This way, the time taken to Hibernate and wake up is reduced and at the same time it does not comprome on future access speeds after waking up as it would have stored the most needed data.
Think about other side effects - Longevity of Disks as super fetch reduces disk activity and less power consumption…
Isnt this cool??? Do you want to listen to the Architect who was behind this - click the below link…
http://channel9.msdn.com/Showpost.aspx?postid=242429
-
Agile! Agile! – Where Art Thou - Part 1
Posted on July 19th, 2007 No commentsThese days there is a lot of hue and cry about a new model of development called “Agile”. The name “Agile” doesn’t give any clue as to the fundamental structure of this methodology unlike Waterfall Model or Spiral Model or Iterative Development Model. Instead the name implies two attractive positive qualities “Faster time to market” and “Friendlier to last minute or continuous changes” much loved by the marketing team. Also there is a hidden incentive for the CFO - shorter dev time, causing may be a higher burn rate, but possibly lesser overall cost and definitely immediate returns. In general people like to get things delivered fast and at the same time retain the capability to postpone decision making. The Agile keyword seems to provide both and hence definitely will behold attention of decision makers who need to deliver results fast.One wonders - So what is the magic in Agile which helps to achieve this? Is there a catch or have we truly figured out the trick behind the “free lunch”?In this quest, when one tries to understand the structure of Agile we see that there are many different flavors of the same thing which bear the name ‘Agile’ unlike a traditional waterfall model of development or iterative development. In general Agile is a loose term referring to different methodologies - SCRUM, XP (Extreme Programming), DSDM (Dynamic Systems Development Method), Agile Up (out of which SCRUM and XP are the more popular ones atleast in the US) which exhibit certain common characteristics.
Some key Agile Methodology characteristics are (derived from Agile Manifesto)
- The individual is more important than process.
- Do not document and convey, just code and convey. A delivery in the form of document is of no use to the client and doesn’t signify value or progress
- Sign only Time and Material contracts - Do away with Fixed bid contracts as what you are bidding for will definitely change completely.
- Do not plan, but just react intelligently and self organize
- Be open to change till the last moment
- Business and Dev team need to interact face to face on a daily basis.
- Keep delivering working software every few weeks
If you want to delve deeper into Agile, one needs to understand Agile blockbusters like SCRUM and XP as the techniques practiced in these methodologies mostly constitute the overall techniques of ‘Agile’ and define ‘Agile’.
SCRUM
This methodology was invented in 1986 by Takeuchi and Nonaka and was later developed by Jeff Sutherland around 1993 for Easel corporation. It was then adapted for software industry in a formal way by Ken Schwaber in 1995. SCRUM derives its meaning from Rugby which is a way to restart the game.
I repeat here the abstract of Ken’s paper (1995) which elucidates the motivation for a new process methodology.
“The stated, accepted philosophy for systems development is that the development process is a well understood approach that can be planned, estimated, and successfully completed. This has proven incorrect in practice. SCRUM assumes that the systems development process is an unpredictable, complicated process that can only be roughly described as an overall progression. SCRUM defines the systems development process as a loose set of activities that combines known, workable tools and techniques with the best that a development team can devise to build systems. Since these activities are loose, controls to manage the process and inherent risk are used. SCRUM is an enhancement of the commonly used iterative/incremental object-oriented development cycle.”
If you listen to “The roots of SCRUM” presentation by Jeff, the initial implementer or SCRUM processes (1993), you would be pleasantly surprised when he confidently states that SCRUM would question and break the historical “Cost-Time-Features” iron triangle. This is the weapon which is currently used by many smaller software companies to enable them to break into impenetrable forts of large scale contract projects. This is the dream these boutique software shops with their power (extreme) programmers tries to realize for their clients. But I do not think this claim can be proved theoretically as you cant deliver all functionality in zero and zero cost. I think Jeff may be trying to indicate that it is possible to deliver with lesser time, more features and possibly at a lower costs.
What was thought as impossible to do in 12 months, the proponents of SCRUM cheerfully advocate can be achieved in 12 weeks with three intermediate deliveries!! But they won’t sign a contract which promises well defined deliverables, but just a loose agreement and commitment to show rapid visible progress at short intervals. From the client’s side he neednt wait for half year wondering what is happening to the money he is spending, but he sees solid results in a month!! If you don’t see those rapid results which are promised in a month or two, the understanding is that the SCRUM experiment is a flop and you could annul the engagement and cut the loss. Please note that there is a thread of trust which underlies this engagement principle. This is something hard to accept, and as Jeff points out - like alcoholics who cannot resist their temptation, it is hard to resist the temptation of creating a fixed legally binding contract. One should leave that temptation and boldly adopt this iterative trust approach to make SCRUM work.
In terms of actual results, as per Jeff, Primavera which followed SCRUM trebled their productivity in 3 years and increased quality (reduced bug count) by 20 times in 3 years. So SCRUM is not only faster and flexible, but better in terms of product quality is a very compelling argument presented by Jeff. The most successful implementation of SCRUM was by Borland which exceeded industry productivity by an insane number.
Cool!! But how?? There should have been lot of good managers to manage and plan the project to make it that successful. Well No. The recipe is what Jeff calls Self Organization. He says “Do not manage people, let them be who they are and allow them to self organize and the software will emerge!”. This guy has gone bonkers one would think.
This difference in management (or rather non-management) approach is neatly explained by Takeuchi and Nonaka (who coined the term SCRUM) in their seminal HBR article “The New New Product Development Game” where they contrast the traditional model of development and the SCRUM model. Instead of the traditional relay race approach they encourage to ‘follow the holistic or “rugby” approach where the team tries to go the distance as a unit passing the ball back and forth’. In a relay race, the baton is passed between specialized teams - Marketing, R&D, and Production. In Rugby kind of an approach a multidisciplinary team is formed which work together, constantly interact from start to finish and the process and product emerges as a result of this constant activity and self organization.
But then wouldn’t it all lead to chaos and anarchy? This is where the SCRUM methodology or rather SCRUM techniques come in to create an atmosphere of controlled chaos, which seems to be an ideal environment to achieve fast, high quality, customer friendly software development.
In the next part in this Agile series, I will delve more into what constitutes those SCRUM techniques and methods which create the controlled chaos atmosphere.