Showing posts with label data mining. Show all posts
Showing posts with label data mining. Show all posts

Jan 24, 2014

Unfathomable data: are you still scratching the surface?

Take for instance images!

If you have reached at the end of Google image search page, while looking for a random object or word like say "chair" and if you think this is it, here we are staring at the last image of  chair in world ever uploaded online and you start having this feeling that lets change the key word to "modern chair" and next  changing it to "modern designer chair" ; probably you are going in the wrong direction, probably you are approaching it too casually.  Online research is a  serious business whether its data or image. Its far easier to get lost or get disorientated in the   convoluted maze of online data while it needs a serious approach and focused objective to reach where you want to.

Just to get the hint of quantum of information that is floating out there freely online, if you are searching for any image for instance on Google you might get few hundred results now if you search using each of those images using this option called "search by image" you will get another few hundred distinct results for each original image now if you use a  combination of "search by image" and "text" -any text of your choice - you will get another thousand results now if you keep changing the text or combination of text you will keep getting different new results even if you don't change the image used in "search by image". Easy, we are not done yet. Now if you keep changing the "choice of  color" offered  by Google search results you will keep getting further unique results. Other option of image search is to use "random" text along with "text of your choice". 

Now another way is to search for photographs instead of images. Are they any different? Yes, and we are not even talking about twitter or instagram photograph;we are talking about photograph search through Google. Millions of photographs out there, all you have to do is to search for your  choice of text or subject with the several dozens extension names by which different brands of cameras save their files.

True, while still looking for chair, you might not get only "chair" in your search results if you search using above methods but possibility is that you might come across things in course of your search which will inspire you to the core and which might give a whole new direction and meaning to your research, which you can call accidental learning. We haven't  talked about ever flowing ever growing image database of flicker, twitter or instagram even. We haven't talked about text search yet.  

Its not just about research, its about the real fun part of it. Just take a dip in the unfathomable ocean of data its addictive. Be a smart netizen.


Join Anoop on Linkedin  
or follow on Twitter @urbanoregional


Apr 1, 2013

Next time you are bullied tell them you are on an exponential learning curve!



Is it "the right time" for education system to introspect on its evaluation and certification process?

You see, you have been bullied every now or than for apparently no good reasons especially in your early growth phase as you can recall from your childhood memories and being bullied even today while you undertake every new challenge. Take a pause and rewind, being bullied for not performing as well as the next bright child in your class or neighborhood, not been able to memories the obvious answers to the obvious questions asked in the endless class tests, being bullied for not performing financially as well compared to people with similar background in life.

Let’s see what's the DNA of this bullying business and has it any significance in the light of the fact that learning and absorption capacity of a person is mostly independent of their childhood performance due to several unknown and very unpredictable environmental influences and opportunities available in one's life as they grow compared to limited and intentionally restricted options available/ provided to them in early childhood. Has it any significance at all in today’s rapidly changing world with emerging technologies, accessible and free resources of knowledge and why this business of bullying should fade away real fast, at least why its high time for the same to happen. Let’s see why the boundaries of education system should and will disappear very soon.

If you will closely observe the peripheral resources of learning available today while tracing the chronological changes in streams of information dissemination and future possibilities of technological infusion in informal learning process which all cumulatively presenting a new and broadened learning process, this informal learning process is getting so inclusive, so fast every passing day that it will make your head spin. These informal learning tools are mostly ignored in education process at present. 


Let’s look at the parallel growth in education system, which apparently seems way behind the real life real time pace of development when it comes to learning process - internet, smart phone, social networking just couple of examples though, you know! Unfortunately none of these wonderful tools of information exchange and knowledge transfer which have become core of everyone's everyday life are in the priority list of education system at all, and neither going to be an area of interest or exploration very soon for them, rather they will make sure that this plethora of life changing tools shouldn't get into the education system in the name of discipline, in the light of childhood vulnerability! Of course you need some kind of regulatory mechanism.

If we go with the current ideology of educational evaluation system then people in their life should be arranged in the exact same order of professional success as they were arranged in their school or graduation class, as evaluated based on their marks gained, based on their knowledge of the curriculum, but take a look around and you will realize it’s not true, it can’t be and it will never be true. The current evaluation system doesn't acknowledge the real potential of individual; they just care about the part of potential confined to their established boundary of curriculum, including peripheral limited activities like cultural and sports participation, ignoring the future environmental influences especially technological ones.

Present educational evaluation system seems somewhat flawed and bit dangerous as well; this whole process of establishing and certifying who has what capacity at the very early stage of individual's growth period while the growth of a person is directly proportional to what kind of opportunities he or she received in post schooling or later part of educational or personal life. Imagine a hypothetical situation in which an institution firmly establishes this fact that some particulate child has capacity of a D+ grade performance in primary or high school level, imagine if he or she could not get through selection process of a good school at secondary or higher secondary level and hence could not further got enrolled in a good university because of that D+ grade. Now imagine, what if that same child becomes extremely knowledgeable and efficient few years down the line after completing primary or high school education possibly due to new environmental influences, due to the latest learning tools and vast knowledge resources available today, what if he or she is the first one to grasped and master the latest tools of learning and research, what if that D+ grade student is technically the soundest candidate for higher secondary or university entrance but couldn't secure admission just because some institution had long back established and authoritatively certified the poor grade of student based on some skewed benchmark. A dream might get devastated due to early certification of knowledge on a discreet fragmented evolution system often within a limited bandwidth of educational framework. Couple of questions for you to answer - Can you justify assessment establishing poor performance of a geek child in cultural and sports activities or visa-versa? Can you justify assessment based on questions asked in class tests, answers of which you can find out in less than 20 seconds in Google? Can you justify bullying of that poor student who does not know the one word answer of that textbook question, when he or she can find out a precise and elaborate answer of that question within seconds while learning much more in the process only if you could have allowed him or her to use their smart phone? What’s the validity of asking such questions today and what is the validity of educational assessment based on such questions today at all? Don’t you think it’s the right time for the education system to introspect?

Mutation of educationsystem due to technological infusion is an emerging area of exploration which educational strategists should focus on. Learning is a constant process, and with the exponentially growing free and accessible resources of knowledge, learning process is bound to be an exponential phenomenon. So, if you are a parent or a teacher or educationalist think twice before bullying your child or student or to those supposedly less knowledgeable than you today, because they are going to come back tomorrow loaded with knowledge and laced with such innovative tools of precise knowledge acquisition which we can’t even imagine today. Today's comparison of two kids has no realistic relation or reflection at all with the actual success in their life in future when they will be at the highly accentuated curve of learning, the only deciding factor would be who shows how much interest in the emerging technologies and sources of learning, who familiarize himself or herself how much and who absorbs how fast and all of this is totally independent from the performance and potential assessment of today.

It’s not just the fault of education system which apparently fails to timely acknowledge the impact and possibility of technological development which has and will always outpace the development of educational system. Education system has its own constraints which we should also acknowledge, which is that they need to thoroughly investigate, assess and validate everything before letting it into the curriculum of education system, they can’t really merge the trending knowledge into the mainstream education without thorough understanding of the trend and its possible impact on the students, young or mature, and by the time they understand the trend and tame it down to the extent to be eligible for the mainstream educational inclusion, that trend becomes outdated, hence this constantly lagging process of chasing new knowledge and hence this outdated dissipation of knowledge through structured education system. And since the benchmark of assessment is outdated, there is grossly wrong process of assessment of students as well. For another example, no matter what and how extensively that particular student knows about some area of his or her interest, it is not going to help him get the good grade unless that knowledge is about one of the couple of subjects specified in the curriculum. That student is supposed to stick to the curriculum diet imposed on them by the institution; apparently every other knowledge is irrelevant in the eyes of education system at least that’s what they think is important to secure a good job at the end of the day. A fundamentally wrong notion of educational performance at least a highly irrelevant model of evaluation of talent in today's world when you are not just solely dependent on schools and universities for your learning, while you are learning wherever you are, whenever you want, in that very moment, learning on your mere wish. In spite of all this free and immense knowledge the existence of educators can’t be negated, what is needed is to have a holistic approach towards talent evaluation and fast inclusion of technological shifts.

This is "Learning on the go" generation and we can’t confine it in the predefined boundary of education, we can’t impose on others what they should learn, rather they should decide what they want to learn and it’s our responsibility to provide them the customized curriculum of their individual interest, education system will increasingly have to be much more flexible, it’s inevitable and its need of the hour. What needs to be done here is that to create a macro framework for the education system, flexible, fast and inclusive enough to quickly notice, sanitize and accommodate the changes occurring in technological field and otherwise, into the mainstream education system. A framework which makes the educational system sensible and smart enough to allow unlocking the true potential of unique individuals rather than mass production.


Mar 23, 2013

Thriving market of cheap design aesthetics.

Have you ever wondered why you still find those similar crude designs around even after decades?



IMG_0119 by karen horton, on Flickr
Design demands Freedom 
Creative Commons Attribution 2.0 Generic License
 Image by  karen horton 

So, will they keep producing those sub-standard design and inexpensive aesthetics just because they have found a comfortable niche market for that? a compromising non demanding market segment which is either unaware of their right to aesthetics, right to own good design or they have accepted this false imposed notion of design dictatorship that a good design and aesthetics is only for well-off segment. The restrictive and monopolistic approach towards design only leaves majority of people frustrated witnessing bad designs and cheap aesthetics scattered all around which is usually propagated due to lack of serious talent and affordable skills in design industry or sometimes carefully established to make you feel inexpensive!  



Nerds For Nature Launch at CFA on Febuar by Nerds For Nature, on Flickr
Strengthening Design  
Creative Commons Attribution-Share Alike 2.0 Generic License
  Image by  Nerds For Nature 

Next time you are travelling in a public transport or while casually strolling through the busy market street, take a serious look around, you might be overwhelmed by the shear abundance of product designs borrowed from your childhood memories, for example look at couple of shoes of those people standing next to you in a subway, chances are, out of ten pairs of shoes you will find one out of this world customized designer pair "a must have it in your wardrobe style", you will see two highly expensive ergonomically designed branded pair, another two pairs from the trending fashion, and another five pairs of shoes with design and aesthetics borrowed from the memories of your distant past, designs borrowed from different time spans of previous couple of decades. It cant just be a coincidence that 40-60% of designed products are still trapped in the evolutionary stage of design, while we already have the best of benchmarks available and established. and it is applicable to almost every type and scale of product available in the market, shoes are just a crude example!



It's the urgent need and responsibility of a designer and a progressive society at large to unlock and libralise the design and aesthetics in a production environment or otherwise, and educate masses towards the same, letting this dissipated knowledge and awareness permeate through the consciousness of otherwise design ignorant and aesthetically suppressed consumers and society, a wind of change which will help make a better ambiance and surrounding whichever part of the world or whichever corner of the city you are in at the moment.

you might like this post on Design Democracy as well called "Deprived of design aesthetics?" 

Nov 16, 2012

Urban Underprivileged segment-what can we do for them?


While you are driving down the busy city lanes sometime of the year someday, from there to somewhere, busy negotiating ruthless traffic, busy minding the gap, minding your own business, busy processing next to-do-list on your mental map, driving with occasional subconscious glimpse of buzzing city with all that glam and beauty and buzz and noise and colors; glimpse of the city with constant familiar flux; suddenly on the footpath, on the next turn, on the traffic junction or on the dim corner of street you spot a poor deprived child, a beggar in patchwork cloth, an old discarded person or a doped fainted soul, often pretentiously indifferent but sometime being concerned it comes to your mind, what can you do for them? Why they are in that state? What opportunities they lacked? What is their future? What is the way out for them?  Is there someone listening to them? Apparently it’s not their first choices to be there in that unpleasant state!

No matter how rich, how well off, how busy you are in your own life you must have come across such instance and have thought at least for once that what can you do for them? There might be policies, might be schemes for their revival but they might not be aware of such things, there might be some poster of welfare scheme to rescue them from that situation posted right at the display board of that bus stop but may be they don’t know how to read, no one told them either, may be they are educated but they lack a caring hand to push them towards a better life, may be the sheer number of them making it difficult for city administrations to deal with them, may the same sheer number of them calls for restructuring of public welfare policies and re-engineering the implementation strategies.

Something somewhere lacking, some lessons to be learned, some immediate rescue actions need to be taken, with a sincere note of hope we need to initiate a discussion on variety of such issues on variety of platforms to come up with some concrete and realistic alternative livelihood solutions. Something which will give them a respectful livelihood and a dignified life! One still feels that it’s more of policy level issue than the monetary one. Policies like customized education, mandatory skill development, upfront plans for aging city population, teaching survival tricks and strategies right from the elementary education up to the higher education ladder and even to the uneducated population, survival from financial breakdown, survival from natural calamity, surviving from personal, professional, physical and medical emergencies, surviving poverty and old age in a dignified manner of course with institutional support, citizen participation and motivation!

We need to make every citizen skilled and able enough to earn their own livelihood even if they already have livelihood resources at disposal, even if they have family and people to take care of at present, at least as a backup livelihood plan so that they don’t end up being abandoned and begging in streets in case of major setbacks, so that at least there offspring don’t spend whole of their life on street relying on other’s mercy. Simply educating 100% of population is not the ultimate solution for urban poverty for that matter poverty of any kind, even formal education will have to be skill oriented from the very beginning; we can keep narrating encyclopedia to them later! Just an argument though, an 8 to 12 year child who has some livelihood skill up his sleeves can survive even flourish in his life picking up right opportunities, if hypothetically he has no other options left, but it might be very difficult for many of children of same age group to deal with such situations with generalized Not-so-skill-oriented mass education which is being transferred to them at school at present. Education system need to include more and more of technical and creative skill and hobby oriented course structure from the very beginning, one can even think of tailor-made customized education for every single student identifying there talent and inclination early in life. Even if they don’t have to use those skills they can peruse them as their hobby or alternative income source in course of time, and we’ll have a pool of incredibly talented, confident, morally and financially elevated citizens. 

Jun 21, 2012

It's been done this way since ages doesn’t make it the right way today!!

Rethinking academic research methodology.

So, you are up to research, thesis, dissertation or something, great!! Let’s consider few  things before we move ahead, considering few establish academic research norms, said or unsaid, which you might like to question, which you might like to challenge, but you won't, may be you can’t, because someone else is running the show.

Let’s understand the anatomy and objective of research and what should be the scope of any academic research for purposes, like thesis, PhD and so on and let’s see if there is any loophole in the well established century old research methodology and why it’s high time that research methodology has to reinvent itself.

What usually happens is that you choose a topic of your expertise and interest, narrow or wide and a mentor or guide as well either to guide you throughout or just because its mandatory to have one as per university norms, whether its architecture, planning or sociology or something else, it’s all the same. Then you start collecting supporting evidence and snippets of information first to validate the selection of research subject and then to back your conclusion. 

If you delve deeper you will realise that any chosen research subject or topic no matter how wide or narrow it is, has a very wide possible scope for exploration which we unjustifiably narrow down due to many reasons. You can have all the freedom to narrow down the subject or research topic itself which you choose but how can you narrow down and limit the research domain yourself and how can you decide in the beginning of research that which external influences to choose which might affects the subject, just because scope of research seems unfathomable to you for that particular subject or topic, just because someone told you to do so, just because your curriculum allows you the freedom to do so. If you are limiting your research to the review of few books on the subject,  selected by your guide who can have his or her own biased inclination towards those particular books or those authors,  if you limit your research only to some good books which are remaining  in your library after best books of the lot which has already been lifted by the academicians and professors long ago,  if you limit yourself to handful of primary sample survey which has its own grave issues and which is bound to give inaccurate picture, and if you limit yourself for any other similar innumerable reasons, you  are doing a big injustice to the research subject.

The issue with age old academic research methodology in context of today is that there always has been a strong stress on authenticity and completeness of content, fixed research structure, rigidity of data structuring, stress on literature and book reviews and first hand data gathering, there always has been submission to the authority of few established individuals in the field and imposition of idiosyncrasy and idealism of mentor or guide, ignoring the massive and continuous stream of relevant information which is available today and which is already growing and going to grow exponentially in the future. In the inertia of age old established research methodologies we are knowingly or unknowingly ignoring the possibilities that today’s technology provides us in conducting our research. Ignoring the technology of data mining and data exploration which they are already using ruthlessly in business and corporate domain is almost absolutely missing in academic research.

Two fundamental truth which majority of academicians today would be highly hesitant to be confronted with, is first that "Billions of half-baked data is much more reliable and contextual than handful of authentic publications when it comes to research and establishing a fact, excluding field of science and mathematics" and second that "Skewed focus towards book reviews and published literature for research purposes tend to glamorizes second hand work and dampen free and creative thinking spirit". There is nothing new and creative in quoting some author on some instance or subject, nothing new in putting someone else's inferences and analogies in your own words in your academic research report. When there is too much focus on structure and format (Thesis, synthesis, analogy, reference, bibliography and all) content starts losing its significance. This prominent conventional research methodology only leads to half baked conclusion at the end of your research span, because of just few numbers of cases studied and fewer perspectives of authors analysed.

Main constraint here is time and it’s up to you how you manage to get optimum exposure in that short span of time to arrive at valid inferences. More intense the exposure more authentic would be the conclusion or outcome. You are given a limited time to conduct and conclude your academic research whether thesis, dissertation or PhD.  You cannot stretch it infinitely. now you zoom in and pick up few published literature on the subject for your research reading pages, first to last, how many of these  books with "authentic" information you would be able to read in that assigned research span of few months or years, five, ten, twenty or fifty? That’s it? Come end of the research span and you are aware of point of view of five or fifty different authors, thinking you have known enough, thinking it’s time to conclude. No matter how very authentic information you have at the end to conclude your research, these are just point of views of handful of published authors out of millions equally talented or more or less out there in the world working, thinking, deliberating on the same subject, who never got the time, inclination, money or right publisher to get their work and thoughts published but who are constantly writing on forums, blogs, constantly tweeting and sharing and so on, may be you also missed out the brilliant thoughts on the subject because it didn’t reach your library shelf, because your guide didn’t recommended it to you, may be you didn’t have enough money to buy that book or eBook or may be because it couldn’t get the attention of the masses due to competition with shear number of literature on the same subject.

It’s high time that students and researchers need to be encouraged to look for unconventional, previously unexplored and untapped streams of data for mainstream academic research by their colleges, universities and most importantly from their mentors and guides in their dissertation and PhD,  sources like massive free and dynamic data from online environment of blogs, discussion forums,  audio-visual data from established or amateur masses, tiny bits of information which individually doesn’t make any sense but which collectively presents fairly recognisable, logical and surprising patterns, and it’s high time that such research methodologies should get mainstream recognition. It’s time that they should shift their fixated attention from literature reviews to data source explorations and analysis, broadening the domain of literature itself.

You see, it would be much more exciting, authentic and wholesome an experience to explore from the dynamic, widespread and crude "sea of knowledge" rather than exploring from few well-maintained but stagnate "resource lakes".


BY- Anoop Jha 

May 27, 2012

Google - Data for sustainable planning!!



Internet has changed the process and pace of planning in the recent years, still leaving immense untapped possibilities and Google has played a crucial role in all this with constantly evolving its interface, data structuring and data liberalisation. Its up to us how sensibly we derive the hidden treasures and pattern of data and use it for better planning, leading to sustainable communities.