MIT高级讲师:从大数据到深数据

简介:

0.jpg

在过去的10个月,我主持参与了麻省理工的“IDEAS中国”项目--一个由30多位中国商界领袖组成的,为期10个月的创新之旅。今年,该项目招收了中国一个主要国有银行的数位高管。这个团队的目标之一,是在大数据和其他相关的颠覆性技术到来时,仍可以重塑他们组织的未来,这也使我更多地接触了解了中国经济。正如阿里巴巴颇有远见的创始人马云所说,“五年后,我们预计人类纪元将由信息技术时代转变为数据技术时代。


但是,“数据技术”时代、“大数据”时代究竟意味着什么呢?现在,它往往意味着谷歌、亚马逊、Facebook和苹果这类大公司,这些我们曾经喜爱现在却越来越怀疑、不信任和恐惧的公司,会在你毫不知情时,收集你的数据,并转卖给其他公司,当你注意到出现在屏幕上的精准投放的网络广告时也就不足为怪了。有趣的是,最初人们对于美国的这些大数据帝国非常积极的看法现已转向欧洲及世界许多其他地方,包括北美。爱德华·斯诺登事件使我们大家都对大数据的误用更加敏感。但是,这只是表面问题,真正的问题更深层


大数据真正的问题在于:我们的感知及思考能力正在逐渐被计算机算法所取代. 虽然一开始我们会觉得大数据很方便, 也很酷, 能给我们带来许多我们想要的服务.但这同时也引发了有关究竟谁拥有大数据, 个人及公民是否有权利选择如何使用有关于自己的数据的争论.


毫无疑问,大数据创造了许多全新的可能性, 但同时我认为我们应该明确地区分开浅层次的大数据与深层次数据所谓浅层次数据,指的是有关别人的数据: 别人说了什么,做了什么。而这几乎正是目前所有大数据所包括的内容.


而深层次数据是帮助个人和社会来认识他们自己的。深层次数据就像一面镜子:它让你认识你自己--无论是作为个人还是作为社区一员。在我过去二十年的职业生涯中,许多团队和机构在我的帮助下进行了一些有重大意义的创新及革命性的变化。我从中所学习到的一点就是:产生革命性变化的关键就在于清楚地认识自己。这就是为什么深层次数据是很重要的。它对未来的机构,我们的社会以及整个世界都非常重要。


但是如今大数据的所作所为往往是相反的:大数据被用来操纵我们的行为,用我们从没想要的广告来对我们狂轰乱炸。表面上大数据用于将人类思维外包给算法,以降低习惯性思维边界内的意识水平。深数据,如果以正确的方式加以研发和成长,可以帮助我们提高认识水平,并将利益攸关方的意识体系转变过来,从对自我系统的认识(我自己的筒仓意识)转变到对生态系统的认识(整体意识)。


让我用两幅画面简单总结一下表面的大数据和深数据的区别:

0

Big data (science 1.0): data that informs about the world (source: A. Oechsner).


0

Deep data (science 2.0): data that helps us to see ourselves (source: A. Oechsner).


从科学1.0到科学2.0的旅程是将科学观察这一笔直的望远镜掰回来到自我观察的过程 - 这个自我指的是我们的个人和集体的统一。


上周我们的闭幕会议结束时,中国国有银行的高层领导反思了自己在过去十个月当中的旅程。每个人都在报告中提到了他们在如何思考和运作方面的深刻转变。这里有两种典型的叙述:


“这次旅行不只是工具和知识。它同时改变着你的思维方式,它可以让你在挑战面前跳出旧的思维模式。我自己在改变,我也觉得我的同事在改变,我们更容易达成共识,我觉得我同事生活做事的目的在转变。因此,我们更注重经验,能够更好地执行。


“对我来说,IDEAS的旅程是思想的旅程。它打开了一种新的思维方式,一种新的关联方法,以及一种新的存在方式。”


从本质上说,IDEAS的参与者都讲到了以下变化:

•思维:从单纯接受老模式,到创造性思维

•对话:从辩论到平和对话

•协作:从自我被动到忘我主动


在过去的这几个月,参加活动的这些高层领导们被分成四个小组,每个小组都试图尝试一些新的方法,来寻求未来的机会。令我吃惊的是,每个小组都开发出一种跨组织的合作平台原型,每个利益方都能通过这个平台用数据进行沟通。所有这些平台的建设原型都还在早期阶段,不过有一点是这四个小组都反复提到的,那就是大家思考问题要从“我”转到“我们”,从“自我”转到“整体”的重要性。


他们的努力就给我留下了这样一个问题:在社会层面上,究竟什么类型的深层数据基础设施能够有助于把这种“细微的个体观察”回归到对整个社会生态系统的水平的观察


例如,今天我们用国民生产总值GDP来衡量经济进步。国民生产总值GDP是社会经济表面数据的一个很好的衡量。但是用什么等效的深数据工具来衡量一个社会真正的经济进步?我相信这样一个衡量系统应该植根于真实的社会发展成果(例如,预期寿命),以及个人和社区(如生活质量)的发展状况。去年Presencing研究所,GIZ全球领导学院(德国发展合作部)和位于Bhutan的国民幸福指数中心联合发起了全球福祉实验室(Global Wellbeing Lab),该实验室把世界各地来自政府,企业和民间社会的领导人联合在一起,来开拓寻找新的指标和深数据工具,帮助社区和社会生态系统观察自己,建立观测衡量社会运营的新模式。


今天你在哪里能看到这样的新的指标体系或深数据的工具在生根发芽?我们可以从这些早期例子中学到什么?深层数据对你自己意味着什么?在你自己的生活和工作中什么是快乐幸福的真实来源,哪些指标可以以更有意义的方式帮你看到和感觉到自己的发展?我们如何才能共同开拓,实现商业,社会及个人从大数据到深层数据的转变?这都是我们要思考的问题。


原文:

Over thepast ten months I have chaired and co-facilitated MIT's IDEAS China program--aten month innovation journey for a group of 30 or so senior Chinese businessleaders. This year the IDEAS China program enrolled executives of a majorstate-owned Chinese bank. One goal of this team was to reinvent the future oftheir organization in the face of big data and other related disruptivechanges, which provided me with a little more exposure to that aspect of theworld economy. For example, Jack Ma, the visionary founder of Alibaba, saysthat "In five years, we anticipate that the human era will move from theinformation technology era to the data technology era."


But whatdoes it mean to be in an era of "data technology" and "bigdata"? Until today, it has often meant that big companies like Google,Amazon, Facebook, and Apple--the same companies that we used to love and nowincreasingly begin to question, mistrust or fear--take your data without askingand sell it to other companies without your knowledge (until you notice thetargeted Web commercials that appear on your screen). I find it interesting thatpeople's initially very positive view of these American big data empires hasbeen shifting first in Europe, but now also in many other parts of the world,including North America. Edward Snowden made all of us more sensitive to themisuse of big data. But that's just the surface issue. The real problem is on adeeper level.


The realproblem of big data is that we are increasingly outsourcing our capacity tosense and think to algorithms programmed into machines. While this seems veryconvenient and cool at first and offers access to services that many of uswant, it also raises a question about who actually owns big data, about therights of individuals and citizens to own their personal data and to exercisechoices regarding its use.


While bigdata has certainly opened up a whole new range of possibilities, I would liketo suggest a distinction between surface big data and deep data. Surface datais just data about others: what others do and say. That is what almost allcurrent big data is composed of.


Deep data isused to make people and communities see themselves. Deep data functions like amirror: it makes you see yourself--both as an individual and as a community.Over the past twenty years of my professional life I have been helping teamsand organizations go through processes of profound innovation andtransformative change across sectors and cultures. The one thing that I havelearned from all these projects is that the key to transformative change is tomake the system see itself. That's why deep data matters. It matters to thefuture of our institutions, our societies, and our planet.


Butwhat happens today with big data often is the opposite: big data is used tomanipulate our behavior, to bombard us with commercials that we never askedfor. Surface big data is used to outsource human thinking to algorithms, toreduce our level of awareness inside the boundaries of habitual thought. Deepdata, if developed and cultivated in the right way, could help us to enhancethe level of awareness and consciousness and to change the system byshifting the consciousness of stakeholders in that system from ego-system awareness(awareness of my own silo) to eco-system awareness (awareness of thewhole).


Letme summarize the distinction between surface big data and deep with two simpledrawings:


Thejourney from science 1.0 to 2.0 is a journey of bending the beam ofscientific observation back onto the observing self--both individually andcollectively.


Atthe end of our closing meeting last week, the senior leaders of the Chinesestate-owned bank reflected on their own journey of the past ten months. Everyone of them reported a profound shift in how they think and operate. Here aretwo exemplary statements


"This journey is not just about tools and knowledge; itshifts your way of thinking and it allows you in the face of challenges to jumpout of the box of old thinking. It feels like my self has been shifting. I alsofelt that shift among my colleagues. We get to consensus more easily. I feelthere is a shift of intention among my colleagues. As a result, we are more intouch with our experience and we are able to execute better."

"To me, the IDEAS journey is a journey of the heart. Itopened a new way of thinking, a new way of relating, and a new way ofbeing."


In essence,what the IDEAS participants described was a transformation of

• thinking: from downloading old patterns tothinking creatively

• conversing: from debate to generativedialogue

• collaborating: from ego-centric/reactiveto more eco-centric and co-creative


Over thepast months, while staying in their jobs, the participants split into fourteams that each tried to prototype some new way of operating in order toexplore future opportunities. What struck me was that each team ended up developinga new platform of cross-organizational collaboration that used data as a toolfor transforming the way their stakeholders communicate. All of theirprototypes are still in an early stage. But one lesson that was mentioned bythe teams repeatedly was the importance of shifting their mindset from meto we, from ego to eco.


The questionthat their efforts have left me with is this: On a societal level, what typesof deep data infrastructures might facilitate this 'bending of the beam ofobservation' back onto the observer on the level of entire eco-systems?


For example,today we use GDP to measure economic progress. GDP is an excellent measure ofsurface data. But what would the equivalent deep data tool be for measuringreal economic progress in a community? I believe that it would include a newindicator system that is grounded in real outcomes (e.g., life expectancy), andin the wellbeing of individuals and their communities (e.g., quality of life).Last year we--the Presencing Institute, with the GIZ Global Leadership Academy(German Ministry for Development Cooperation) and the Gross National HappinessCentre in Bhutan-- launched the Global Wellbeing Lab,The lab links leaders from government, business, and civil society around theworld who are pioneering new indicators and deep data tools that helpcommunities and eco-systems to see themselves, in order to sense,andprototype new ways of operating.


Where areyou seeing the seeds of such new indicator systems or deep data tools today?What can be learned from these first examples? What would deep data mean foryour self? What are the real sources of well-being and happiness in your ownlife and work and what metrics could help you to see and sense your owndevelopmental path in a more meaningful way? How can we co-pioneer the shiftfrom big data to deep data in business, society and self?



原文发布时间为:2014-07-25

本文来自云栖社区合作伙伴“大数据文摘”,了解相关信息可以关注“BigDataDigest”微信公众号

相关实践学习
简单用户画像分析
本场景主要介绍基于海量日志数据进行简单用户画像分析为背景,如何通过使用DataWorks完成数据采集 、加工数据、配置数据质量监控和数据可视化展现等任务。
SaaS 模式云数据仓库必修课
本课程由阿里云开发者社区和阿里云大数据团队共同出品,是SaaS模式云原生数据仓库领导者MaxCompute核心课程。本课程由阿里云资深产品和技术专家们从概念到方法,从场景到实践,体系化的将阿里巴巴飞天大数据平台10多年的经过验证的方法与实践深入浅出的讲给开发者们。帮助大数据开发者快速了解并掌握SaaS模式的云原生的数据仓库,助力开发者学习了解先进的技术栈,并能在实际业务中敏捷的进行大数据分析,赋能企业业务。 通过本课程可以了解SaaS模式云原生数据仓库领导者MaxCompute核心功能及典型适用场景,可应用MaxCompute实现数仓搭建,快速进行大数据分析。适合大数据工程师、大数据分析师 大量数据需要处理、存储和管理,需要搭建数据仓库?学它! 没有足够人员和经验来运维大数据平台,不想自建IDC买机器,需要免运维的大数据平台?会SQL就等于会大数据?学它! 想知道大数据用得对不对,想用更少的钱得到持续演进的数仓能力?获得极致弹性的计算资源和更好的性能,以及持续保护数据安全的生产环境?学它! 想要获得灵活的分析能力,快速洞察数据规律特征?想要兼得数据湖的灵活性与数据仓库的成长性?学它! 出品人:阿里云大数据产品及研发团队专家 产品 MaxCompute 官网 https://www.aliyun.com/product/odps 
相关文章
|
2月前
|
分布式计算 DataWorks IDE
MaxCompute数据问题之忽略脏数据如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
47 0
|
2月前
|
SQL 存储 分布式计算
MaxCompute问题之下载数据如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
38 0
|
2月前
|
分布式计算 关系型数据库 MySQL
MaxCompute问题之数据归属分区如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
35 0
|
3月前
|
机器学习/深度学习 存储 分布式计算
机器学习PAI关于maxcompute上用protobuf 处理数据,比较方便的方式
机器学习PAI关于maxcompute上用protobuf 处理数据,比较方便的方式
|
2月前
|
分布式计算 DataWorks BI
MaxCompute数据问题之运行报错如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
38 1
|
4月前
|
存储 SQL 大数据
dataCompare大数据对比之异源数据对比
dataCompare大数据对比之异源数据对比
102 0
|
2月前
|
分布式计算 关系型数据库 数据库连接
MaxCompute数据问题之数据迁移如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
33 0
|
2月前
|
分布式计算 Cloud Native MaxCompute
MaxCompute数据问题之没有访问权限如何解决
MaxCompute数据包含存储在MaxCompute服务中的表、分区以及其他数据结构;本合集将提供MaxCompute数据的管理和优化指南,以及数据操作中的常见问题和解决策略。
38 0
|
4天前
|
数据采集 搜索推荐 大数据
大数据中的人为数据
【4月更文挑战第11天】人为数据,源于人类活动,如在线行为和社交互动,是大数据的关键部分,用于理解人类行为、预测趋势和策略制定。数据具多样性、实时性和动态性,广泛应用于市场营销和社交媒体分析。然而,数据真实性、用户隐私和处理复杂性构成挑战。解决策略包括数据质量控制、采用先进技术、强化数据安全和培养专业人才,以充分发挥其潜力。
11 3
|
7天前
|
运维 供应链 大数据
数据之势丨从“看数”到“用数”,百年制造企业用大数据实现“降本增效”
目前,松下中国旗下的64家法人公司已经有21家加入了新的IT架构中,为松下集团在中国及东北亚地区节约了超过30%的总成本,减少了近50%的交付时间,同时,大幅降低了系统的故障率。