大数据工程应用 数据分析

It’s been quite an adventure since the dawn of my big data engineer career. I started out differently compared to a typical path. Being a fresh graduate, I possessed nothing more than an academic research background and an enthusiastic attitude. But that wasn’t enough, and I’ve had a couple of rough years.

自从我的大数据工程师职业生涯来临以来,这真是一次冒险。 与典型路径相比,我的开始方式有所不同。 作为一名应届毕业生,我只具有学术研究背景和热情的态度。 但这还不够,我经历了几年的艰难时期。

In life, things turn out to be much simpler when you know what to avoid. I didn’t have that luck, but I’ve learned over my mistakes. There’s a saying in my culture that goes: “You don’t get to choose your job, but it picks you instead”. It was somewhat true in my case. I did not choose big data engineering in the first place.

在生活中,当您知道该避免什么时,事情就会变得简单得多。 我没有那么幸运,但是我已经从错误中学到了东西。 在我的文化中有句谚语说: “您没有选择工作,而是选择了您” 。 就我而言,这是正确的。 首先,我没有选择大数据工程。

In life, things turn out to be much simpler when you know what to avoid

在生活中,当您知道要避免什么时,事情变得简单得多

I committed a lot of errors, chose many wrong paths, and put my faith in some wrong hands. But I’m happy where I am today, I’m willing to pour my heart out to let you know how foolish I was before becoming a big data engineer. There’s nothing to brag about, just my story and what you can get out of it.

我犯了很多错误,选择了许多错误的道路,并将我的信念交到了错误的手里。 但是,我对今天的状况感到很高兴,我愿意为成为大数据工程师之前的工作倾尽全力。 没有什么值得吹嘘的,只有我的故事以及您可以从中学到的东西。

Web开发开始 (Web development as a beginning)

I was eager to land a job, any job. I did some computer programming kind of stuff here and there back in college. Being an immigrant, securing a job is my way to settle down in a foreign country. I wasn’t exactly confident about my technical competence nor professional communication skills.

我渴望找到一份工作,任何一份工作。 我在大学里到处都做过一些计算机编程的工作。 作为移民,找到工作是我在国外定居的方式。 我对自己的技术能力或专业的沟通技巧并不十分自信。

Nevertheless, I began my process as everyone else does: I was spamming my job application. I picked up phone calls, suited up, and went for some of the first official interviews of my life. Things were hard. People turned me down rather than call me back. It didn’t demotivate me as it should, but I started to doubt myself.

但是,我像其他所有人一样开始了我的工作:我向工作申请表发送垃圾邮件。 我打了个电话,适应了我的生活,并进行了我人生中的一些首次正式面试。 事情很难。 人们拒绝了我,而不是给我回电话。 它并没有使我动力不足,但我开始怀疑自己。

Photo by Saulo Mohana on Unsplash
Saulo Mohana在Unsplash上拍摄的照片

I was frustrated. I started to expand my job search to the ones that even slightly related to my study. I initially wanted to land a job as a data scientist or machine learning engineer. But with my zero experience on the fields, that sounded like a mission impossible. Hopefully, there was one startup that accepted my application as a web developer. Compared to my peers, I was underpaid, but I didn’t hesitate for a second. I needed a job, and I took what was offered.

我很沮丧。 我开始将求职范围扩大到与我的学习略相关的职位。 我最初想找到一名数据科学家或机器学习工程师的工作。 但是,根据我在战场上的零经验,这听起来像是不可能完成的任务。 希望有一家初创公司接受我的应用程序作为Web开发人员。 与同龄人相比,我的薪水低,但我毫不犹豫。 我需要一份工作,然后接受了提供的工作。

This was my first mistake. Don’t take me wrongly. The job was fine, everyone seemed friendly, and the tasks weren’t complicated. It just didn’t live up to what I expected I would be doing once I got out of school.

这是我的第一个错误。 不要误会我。 工作很好,每个人看起来都很友好,而且任务并不复杂。 刚毕业,就没有达到我的期望。

I thought I would be cracking some fancy algorithms on a machine learning model. In reality, I was building websites that had nothing to do with probability and statistical learning. I have no offense towards the web development career itself. We weren’t just meant for each other.

我以为我会在机器学习模型上破解一些花哨的算法。 实际上,我正在构建与概率和统计学习无关的网站。 我对网络开发事业本身没有任何冒犯。 我们不只是为了彼此。

I started to slack off on the job. Instead of figuring out a way with my superior, I chose something else: looking for another job. It sounds bad I know. I was on the job only for a couple of months, but the monotony of a workday was eating me up. I was scared, shameful, and too baffled to talk to anyone. I was doing some quick interviews behind everyone’s back, and I landed another job.

我开始放松工作。 我没有与上司想办法,而是选择了其他东西:寻找另一份工作。 我知道这听起来很糟糕。 我只工作了几个月,但一个工作日的单调使我吃饱了。 我很害怕,可耻,也很困惑,无法与任何人交谈。 我在每个人的背后进行了一些快速面试,然后我找到了另一份工作。

The new job was about data mining, big data, machine learning, every big keyword that you can think of. Needless to say, I was hyped to start my new job right away. I tried to tell my manager that I want to quit with a straight face. He wasn’t shocked, which shocks me that he didn’t. Anyway, we figured out the paper works in good terms. I got a 2-day leave notice. I couldn’t come up with a reason for a farewell party, since not everyone knew of my presence.

新工作是关于数据挖掘,大数据,机器学习以及您能想到的每个大关键字。 不用说,我被炒作立即开始新工作。 我试图告诉我的经理,我要坦率地辞职。 他没有感到震惊,这让我震惊,他没有。 无论如何,我们认为论文工作得很好。 我有两天的请假通知。 我无法提出告别聚会的理由,因为不是每个人都知道我的存在。

On my last day at my first job, I had to say goodbye to one particular colleague that I was close to. I said: “I had to go”. She was surprised: “What do you mean? Where are you going?”. I’m not good at saying goodbye, so I was being direct: “I’m going to quit the company”. She went from surprise to anger, and then a little pity.

在第一份工作的最后一天,我不得不向一个与我接近的特定同事道别。 我说:“我必须走了”。 她感到惊讶:“你是什么意思? 你要去哪里?”。 我不擅长说再见,所以我很直接:“我要辞职了”。 她从惊讶变成愤怒,然后有点可惜。

It was harsh. I didn’t properly handle the job in a professional and emotional way. It was due to my ignorance and naivety that altered my decision. My next hope laid on the next promising job. But little did I know that things just started to get worse on me.

太厉害了 我没有以专业和情感的方式正确地处理这份工作。 由于我的无知和天真改变了我的决定。 我的下一个希望寄托在下一个有希望的工作上。 但是我几乎不知道事情开始变得更糟了。

What you can learn:- You don't have to accept any offers just because it's the only choice. You'll end up regret for it most of the time.- Communication is the key.- The grass might be greener on the other side, but it's merely a matter of perspective.

机器学习仍然是炒作 (Machine learning is still a hype)

I’ve got no time to regret my quitting decision because I’m thrilled to get on the next adventure. In this new job, they put me on a new project that no one has been working on, technically. These kinds of newish projects could have many problems. They are usually vague, lack of technical specifications and they turn out to suck.

我没有时间后悔自己的决定,因为我很高兴能踏上下一次冒险之旅。 在这项新工作中,他们使我从事了一个新的项目,该项目在技术上从未有人进行过。 这些类型的新项目可能会有很多问题。 它们通常是模糊的,缺乏技术规范,结果很糟。

As for this project of mine, it had all those three. They promised me on data mining and machine learning applications, but there was nothing to apply on. No data, no software architecture, no technical vision. I spent my whole day attending meetings where people argue rather than work. I didn’t see anywhere I can use my technical competences to my current position.

至于我的这个项目,它具有所有这三个。 他们向我保证要进行数据挖掘和机器学习应用程序,但是没有什么可应用的。 没有数据,没有软件架构,没有技术视野。 我整天都在参加人们争论而不是工作的会议。 我没有看到任何地方可以利用我的技术能力来担任目前的职位。

Photo by Andrea Piacquadio from Pexels
Pexels的Andrea Piacquadio 摄

My manager was unbearable. At some point, he gave me irrelevant tasks just to keep me busy because neither did he have any idea where we were heading to. That’s when I realized the majority of companies claimed to change the world with Machine Learning and Big Data don’t actually have the proper expertise on the job.

我的经理无法忍受。 在某个时候,他给了我无关紧要的任务只是为了让我忙,因为他也不知道我们要去哪里。 从那时起,我意识到大多数声称通过机器学习改变世界的公司实际上都没有适当的专业知识。

I stayed in the company for a while. Mostly because I wanted to legalize my stay in the country. I forgot to mention that ever since I began my professional career, I’ve never stopped my self-learning routine. Especially through difficult times, you’d better be prepared. I’ve jumped into every single piece of advice on how to be updated with the newest trend, and which skills I should focus the most.

我在公司呆了一段时间。 主要是因为我想合法化在该国的居留权。 我忘了提到,自从我开始职业生涯以来,我从未停止过自我学习的习惯。 尤其是在困难时期,您最好做好准备。 关于如何使用最新趋势进行更新以及我应该重点关注的技能,我已经提出了每条建议。

My ultimate path was still ambiguous. I enhanced my portfolio in the most generic way as possible. That was a common error for a junior engineer. But it was hard to avoid. You won’t know what you want to do when you haven’t tried every option. Then came the day when my employer and I couldn’t bear each other much longer. We said goodbye, but not on good terms.

我的终极道路仍然模棱两可。 我以最通用的方式增强了我的投资组合。 对于初级工程师来说,这是一个常见的错误。 但是很难避免。 如果没有尝试所有选项,您将不知道要做什么。 然后是我和我的雇主再也不能忍受彼此的日子了。 我们说了再见,但条件不好。

I hate myself until this day. I let some buzzwords got into my way of pursuing a solid engineering career. I’ve made a promise to never judge a book by its cover anymore. For once, luck has been on my side. This time I would enter into a new world: the Big Data landscape.

我讨厌自己直到今天。 在我从事扎实的工程职业的道路上,我加入了一些流行语。 我已承诺不再以封面来评判这本书。 有一次,运气一直站在我这一边。 这次,我将进入一个新世界:大数据领域。

What you can learn:- Don't believe on what you hear companies say, but on what you see they do.- Buzzwords are real, and don't fall into traps.

我如何在不知道sh * t的情况下对大数据说是 (How I said yes to Big Data without knowing sh*t)

Remember when Richard Branson said:

记得当理查德·布兰森(Richard Branson)说过的话:

If someone offers you an amazing opportunity and you’re not sure you can do it, say yes — then learn how to do it later.

如果有人为您提供了绝佳的机会,但您不确定自己能否做到,请说“是”,然后再学习如何做。

It might sound cliché but it was true for me once. My Big Data’s encounter happened when I got a job offer to become a Big Data developer. They opened up with me about some requirements on the job that I wasn’t qualified for. Luckily they asked me whether I was ready to enter into an uncertain world of Big Data. Things like Scala or Hadoop ecosystem were distant to me, but I said yes. It was not a confident “yes” as in:” Yes I can do it!”, but rather a what-the-hell-could-go-wrong “yes”.

听起来有些陈词滥调,但对我来说曾经是真的。 我获得大公司成为大数据开发人员的工作时,就遇到了我的大数据。 他们向我公开了一些我没有资格胜任的工作要求 。 幸运的是,他们问我是否准备好进入不确定的大数据世界。 像Scala或Hadoop生态系统之类的东西对我来说很遥远,但我说是的。 这不是一个自信的“是”,例如:“是,我可以做到!”,而是一个错误的 “是”。

Then my adventure in the Big Data world commenced. I started out by skimming some books on Hadoop and Spark — two of the biggest concepts on Big Data at that moment. I managed to comprehend their architecture and technical background, but it was tough to grasp the tools without using it. As I waited long enough to learn on the job, I launched myself forwards the mysterious territory of Big Data.

然后,我在大数据世界中的冒险开始了。 我首先浏览了有关HadoopSpark的一些书籍,这是当时有关大数据的两个最大概念。 我设法理解了他们的体系结构和技术背景,但是不使用它就很难掌握这些工具。 当我等待足够长的时间来学习这份工作时,我开始着手转发大数据的神秘领域。

What it feels like entering the Big Data world. Photo by Ashley Batz on Unsplash
进入大数据世界的感觉。 图片由Ashley Batz在Unsplash上拍摄

At first, I was amazed by how data didn’t behave in an ordinary way as for small data. The rules were particular here in this domain, and I have to get used to it. But as soon as I understood how things work, I’ve realized there was nothing special about Big Data. I told myself: “It is just big. As an engineer, we merely use the tools to process big data as the way it is, no more no less”. God, I was completely wrong back then.

起初,我对数据没有像小数据那样以常规方式表现感到惊讶。 规则在此领域中特别重要,我必须习惯它。 但是,一旦我了解了事情的运作方式,我就意识到大数据并没有什么特别之处。 我对自己说:“它很大。 作为工程师,我们仅使用工具按原样处理大数据,这同样如此。” 天哪,那时候我完全错了。

Allow me to cite another saying from my native language

请允许我用母语说另一句话

A frog at the bottom of a well regards the world by the size of the well’s circumference.

井底的青蛙通过井的周长大小来关注世界。

I was that frog. I thought Big Data is so small that I could understand it all. I forgot that I was at the edges of the Big Data realm, and my journey has barely begun. Big Data engineering is not only about leveraging tools to process data at scale, but also how we handle this precious resource to bring values into business decision making and client-facing presentations.

我就是那只青蛙。 我认为大数据太小了,以至于我无法理解所有内容。 我忘记了自己处于大数据领域的边缘,而我的旅程才刚刚开始。 大数据工程不仅涉及利用工具大规模处理数据,还涉及我们如何利用这一宝贵资源将价值带入业务决策和面向客户的演示中。

What you can learn:- You can start woking professionally on a field without minimal experience, but that doesn't exists everywhere. Be prepared.- Be humble. What you know today might be obsolete tomorrow.

结论 (Conclusion)

Estée Janssens on 雅诗兰黛森斯上UnsplashUnsplash

I’ve listed my errors so you won’t commit them again. A professional career path is long, rigid, and somewhat tedious. We progress quicker with experience, and also by avoiding mistakes made by the ones before us. I hope that when it’s your turn, the journey will be as well adventurous, full of challenges, and fruitful at the end. Until then, happy learning!

我已列出我的错误,因此您不会再次提交。 专业的职业道路漫长,僵化且有些乏味。 我们会凭借经验以及避免遇到的人犯的错误而更快地进步。 我希望轮到您了,这个冒险之旅同样充满冒险,充满挑战,并且最终会硕果累累。 在那之前,学习愉快!

My name’s Nam Nguyen, and I write (mostly) about Big Data. Enjoy your reading? Follow me on Medium and Twitter for more updates.

我叫Nam Nguyen,(主要)写有关大数据的文章。 喜欢阅读吗? 在Medium和Twitter上关注我以获取更多更新。

翻译自: https://towardsdatascience.com/my-way-to-big-data-engineering-9f681dc6e140

大数据工程应用 数据分析


http://www.taodudu.cc/news/show-2558343.html

相关文章:

  • 视觉 数据_视觉数据讲故事的力量
  • 利用python进行数据分析(第二版)_第六章
  • 数据科学数据清理和可视化,适合使用python的初学者
  • BEGAN(Boundary Equilibrium GenerativeAdversarial Networks)-pyTorch实现
  • 大数据可视化设计师丹尼斯_自助数据可视化设计师如何谋生
  • 数据科学数据分析_使用数据科学进行营销分析
  • 什么是Usenet | 第2部分
  • 数据铁笼-郑州Android项目ViewPager刷新问题
  • MySQL中文存到数据库是,springMVC保存数据到mysql数据库中文乱码问题解决方法
  • gan处理自己的数据集_数据集难找?GAN生成你想要的数据!!!
  • 登峰造极__边界均衡生成对抗网络(BEGAN)(七)
  • 小试牛刀__GAN实战项目之mnist数据集(二)
  • Python数据分析学习系列 六 数据加载、存储与文件格式
  • 什么事数据补全_事无补
  • 税收和成本计算器php,企业税收计算器的使用方便吗?
  • C++实验3个人所得税计算器
  • 税收学考试可以带计算机吗,税务师考试能带计算器和草纸吗?简答题需要用笔吗?...
  • 第九周上机项目3:个人所得税计算器
  • 用计算机税额怎么计算,企业所得税计算器(企业税收怎么算计算器)
  • 10-税收计算器
  • c++实验3——个人税收计算器
  • DA转换器原理及应用(报告)
  • 推荐几款2021好用的可视化报表工具
  • 完全java实现一款开源的报表工具简表(JOR)
  • EasyReport报表工具
  • Davinci BI报表工具~
  • 推荐几款流行的开源报表工具
  • 一款简单易用的web报表工具
  • java定制化报表_定制自己的报表!7款实用开源报表工具
  • python开源报表工具_12个最好的开源报表工具

大数据工程应用 数据分析_大数据工程的方式相关推荐

  1. 系统的认识大数据人工智能数据分析中的数据

    今天,大量数据.信息充斥我的日常生活和工作中,仿佛生活在数据和信息的海洋中,各类信息严重影响了我们的生活,碎片.垃圾.过时信息耗费了我们宝贵时间,最后可留在我们大脑中的数据.信息和知识少之又少,如何提 ...

  2. 大数据与数据分析:大数据开发岗和分析岗对比

    近几年的大数据,确实在行业当中得到越来越多的重视,越来越多的企业开始成立数据业务部门,针对企业不断累积起来的数据资产,进行价值挖掘和应用.对于企业而言,大数据相关人才的引进,有大数据开发,也有数据分析 ...

  3. 基于python家庭用电量数据的数据分析可视化 完整数据可直接运行

    项目演示:基于python的家庭用电量数据的数据分析可视化 完整数据评论区自取_哔哩哔哩_bilibili 本博客附完整的数据代码: from numpy import nan from pandas ...

  4. teablue数据分析_大数据时代来临,再不抓住风口你就out了

    在大数据时代,数据分析师这个职业似乎应运而生,作为一名数据分析师,我觉得我有必要向大家科普一下这个职业.下面是我的数据分析师证书. 首先数据分析师其实分为两类,一类是业务数据分析师,一类是大数据分析师 ...

  5. Hadoop! | 大数据百科 | 数据观 | 中国大数据产业观察_大数据门户

    深度好文丨读完此文,就知道Hadoop了! 来源:BiThink 时间:2016-04-12 15:14:39 作者:陈飚 "昔我十年前,与君始相识." 一瞬间Hadoop也到了要 ...

  6. 大数据 就业 缺口_大数据专业就业前景 人才需求缺口大 今年增了24倍

    站在互联网风口上的"大数据",已经成为当前社会的热词.继2016年北京大学.中南大学.对外经贸大学首批设立大数据相关学科,中国人民大学.北京邮电大学.复旦大学.重庆理工大学在内的3 ...

  7. 大数据职业理解_大数据带给我们职业三大根本改变

    那么,大数据为什么成为所有人关注的焦点?大数据带来了什么样的本质性改变?为此,我们与中国计算机学会大数据学术带头人.中国人民大学信息学院院长杜小勇教授进行了访谈. 大数据(Big data)通常用来形 ...

  8. 什么叫大数据人物画像_大数据时代,如何构建精准用户画像,直击精细化运营...

    移动互联网时代,精细化运营逐渐成为企业发展的重要竞争力,"用户画像"的概念也应运而生.用户画像是指,在大数据时代,企业通过对海量数据信息进行清洗.聚类.分析,将数据抽象成标签,再利 ...

  9. 大数据 挑战 机会_大数据可视化面临哪些挑战

    数据可视化在大数据场景下面临诸多新的挑战,包括数据规模.数据融合.图表绘制效率.图表表达能力.系统可扩展性.快速构建能力.数据分析与数据交互等. 数据规模 大数据规模大.价值密度降低,受限于屏幕空间, ...

  10. python大数据就业方向_大数据专业未来的就业方向有哪些?[黑马程序员]

    目前大数据是一个十分热门的行业.一个行业的兴起必定会衍生出相对应的岗位.本篇文章给大家总结下学习大数据后未来可以就业的方向,给大家提供参考. 在数据驱动的未来,大数据人才市场势必会越来越大,而现在仅仅 ...

最新文章

  1. MySQL列的别名 insert into select from
  2. 什么是一致性 Hash 算法
  3. 入门|机器学习中常用的损失函数你知多少?
  4. C++使用命名空间中成员的三种方式
  5. 01-VMware-workstation14安装
  6. CISP人才年薪30W起步优先考虑它
  7. 英语语法---分词短语详解
  8. 《深入浅出数据分析》第十三章
  9. 从1到N迈向从0到1:华为创新理念升级详解
  10. 白鹭引擎写入文字图层方法实例
  11. android sqlite使用之模糊查询数据库数据的三种方式
  12. 【华为云技术分享】处理器存储模型概述(2)
  13. spring boot guava cache 缓存学习
  14. [C++] C/C++ 取整函数ceil(),floor()
  15. 关于批判性思维(Critical Thinking)
  16. 微信公众号 模板消息 定时推送 java
  17. 结合P2P软件使用Ansible分发大文件
  18. 吴恩达老师深度学习课程完整笔记
  19. 电脑双屏有一个黑屏_电脑显示器 黑屏(只应对其中一个原因的方案)
  20. IDV软件相关中英文对照

热门文章

  1. html制作简历供人填写,求职简历制作个人定制代写
  2. Spring AOP 循环应用(This means that said other beans do not use the final version of the bean..)
  3. 清华梦的粉碎-写给清华大学的退学申请
  4. PPP和PDP激活区别
  5. 计算机网络传输速度单位换算,网络速度单位换算
  6. 官方rom提取原签名工具_从安卓官方版ROM中提取你喜欢的软件(单独安装包)
  7. 【编程题】【Scratch一级】2022.03 飞翔的小猫
  8. 新浪微博热门话题(30 分)(字符串)
  9. 高考录取查询显示内部服务器错误,为什么江苏成人高考成绩查询网页一直打不开...
  10. Java 结构化数据处理开源库 SPL