g********s
发帖数: 3652 | 1 【 以下文字转载自 DBA_BI_Club 俱乐部 】
发信人: greenlands (sunflower), 信区: DBA_BI_Club
标 题: 聊Data Warehouse 2. ETL
发信站: BBS 未名空间站 (Sun Mar 16 20:22:29 2014, 美东)
Data Warehouse里的 tables 主要分为两大类：ETL, reporting
这贴只聊ETL tables 。 ETL=Extract, transform, load. 这是干什么呢？
Data Warehouse 的数据来自不同部门。比如银行的来自：
- 投资产品部门（你买的各种 funds）
- 贷款部门 （房屋贷款，汽车贷款，商业贷款）
- 存款账户 （saving acct, checking acct...）
这三个部门分别有自己的数据系统，数据不统一，把它们混合在一起进入一个Data
Warehouse就要规范化。我现在说得简单，干起来还需要了解一些银行业务的细节，当
然你会有 business analysts 来解释，他们已经有文档供你读。
ETL发展过程中有过... 阅读全帖 |
|
x*n
发帖数: 33 | 2 【 以下文字转载自 JobHunting 讨论区 】
发信人: xwn (xwn), 信区: JobHunting
标 题: 求问Facebook的data scientist ETL interview
发信站: BBS 未名空间站 (Sat Jan 31 22:24:00 2015, 美东)
有一个2选1的interview
SWE or ETL
如果SWE可以的话就去当码农了。所以只能选ETL。
这个是recruiter 给的 ETL interview的“定义”。有没有人能给出个具体点的例子。
谢谢！
1 person ETL (Extract, Transform, Load):
ETL interview is where you solve a complex close-ended data manipulation
problem using code (typically SQL). An ETL interview will have a clear “
input and output” expectation. *Being able to set up ... 阅读全帖 |
|
x*n
发帖数: 33 | 3 【 以下文字转载自 JobHunting 讨论区 】
发信人: xwn (xwn), 信区: JobHunting
标 题: 求问Facebook的data scientist ETL interview
发信站: BBS 未名空间站 (Sat Jan 31 22:24:00 2015, 美东)
有一个2选1的interview
SWE or ETL
如果SWE可以的话就去当码农了。所以只能选ETL。
这个是recruiter 给的 ETL interview的“定义”。有没有人能给出个具体点的例子。
谢谢！
1 person ETL (Extract, Transform, Load):
ETL interview is where you solve a complex close-ended data manipulation
problem using code (typically SQL). An ETL interview will have a clear “
input and output” expectation. *Being able to set up ... 阅读全帖 |
|
t*******e
发帖数: 127 | 4 大家好，我下星期将要去FB onsite面DS职位，五轮，其中两轮是Product/Analytics，
一轮Math/Probability，一轮SQL，一轮ETL。前四轮都很好理解，网上资料也很多。唯
独ETL我不太理解会怎么考察,已经与SQL那轮有什么区别。我现在做的是analyst，data
engineer/warehouse的那块工作并没有太多接触，所以对ETL没有太多了解，也不清楚
FB会怎么考。我把HR发给我的邮件关于ETL的描述贴在下面，希望有过类似经历的同学
能给我一些指点，比如说怎么准备，样题等等，谢谢！
1 person will focus on ETL -
ETL - This is a technical interview is where you solve a complex close-ended
data manipulation problem using code (typically SQL). An ETL interview will
have a clear “input and output” expectation.
Concep... 阅读全帖 |
|
x*n
发帖数: 33 | 5 有一个2选1的interview
SWE or ETL
如果SWE可以的话就去当码农了。所以只能选ETL。
这个是recruiter 给的 ETL interview的“定义”。有没有人能给出个具体点的例子。
谢谢！
1 person ETL (Extract, Transform, Load):
ETL interview is where you solve a complex close-ended data manipulation
problem using code (typically SQL). An ETL interview will have a clear “
input and output” expectation. *Being able to set up an efficient pipeline
for a problem is the goal for ETL. |
|
P******G
发帖数: 86 | 6 Paypal data management team is looking for ETL consultants (12 months
contract, with possibility of extending to 24 months or convert to full time
) in its Austin office. I simplified the requirement and took out a lot of
formality sentences.
Requirement for ETL consultant:
At least 3 years working with Oracle database.
Familiar with database technologies in general.
At least 2 years with Informatica ETL tool
knowledge about data warehouse is a plus
Requirement for senior ETL consultant:
At lea... 阅读全帖 |
|
b***e
发帖数: 39 | 7 【 以下文字转载自 JobHunting 讨论区 】
发信人: babie (我是养纯种草履虫的贵族), 信区: JobHunting
标 题: Job Openings: Data Warehouse Architect/ETL/Business Objects Architect/Developer
发信站: BBS 未名空间站 (Mon Nov 8 14:52:59 2010, 美东)
Our company is a global fortune 100 located in Midwest having several IT
positions available. These positions are not entry-level positions. Please
email me with your resume or if you have any questions at dyadmstaffing@
gmail.com if you are interested in any of them. These are exempt full time
posi... 阅读全帖 |
|
b***e
发帖数: 39 | 8 Our company is a global fortune 100 located in Midwest having several IT
positions available. These positions are not entry-level positions. Please
email me with your resume or if you have any questions at dyadmstaffing@
gmail.com if you are interested in any of them. These are exempt full time
positions with competitive pay. We will offer relocation,a complete benefit
package, including 401K/ESOP, pension, health, life and dental insurance.
Data Warehouse Architect
We are looking for a hard cor... 阅读全帖 |
|
l******t
发帖数: 660 | 9 【 以下文字转载自 Database 讨论区 】
发信人: lakewest (lake), 信区: Database
标 题: 有个ETL data engineer的opening
发信站: BBS 未名空间站 (Mon Dec 2 15:59:57 2013, 美东)
主要做mysql到data warehouse的 ETL, 一些BI reporting, 也可能involve开始一些
big data hadoop的project， 要求有一定的sql 基础， ETL的概念， java
programming的基础
地点在phoenix, az, 有兴趣的站内信 |
|
R*****n
发帖数: 355 | 10 请写明你想申请的职位， email简历至[email protected]/* */
-----------------------------------------------
1, Access/SQL
Financial/Banking in Minnetonka, MN
Our client is looking for someone that loves data! This candidate’s ideal
background for this role would be SQL, ETL, Data Warehouse, Access, and
business intelligence in their background. The client needs someone with a
change management and price improvement mindset. This person must support
best practices and will be joining a team that will con... 阅读全帖 |
|
l******t
发帖数: 660 | 11 【 以下文字转载自 Database 讨论区 】
发信人: lakewest (lake), 信区: Database
标 题: 有个ETL data engineer的opening
发信站: BBS 未名空间站 (Mon Dec 2 15:59:57 2013, 美东)
主要做mysql到data warehouse的 ETL, 一些BI reporting, 也可能involve开始一些
big data hadoop的project， 要求有一定的sql 基础， ETL的概念， java
programming的基础
地点在phoenix, az, 有兴趣的站内信 |
|
Y*******n
发帖数: 4646 | 12 【 以下文字转载自 Military 讨论区 】
发信人: YourQueen (Her Highness), 信区: Military
标 题: 借宝地求：谁会oracle retail etl script? (转载)
发信站: BBS 未名空间站 (Thu May 30 10:46:30 2013, 美东)
请版主保留一会， 谢谢
发信人: YourQueen (Her Highness), 信区: Stock
标 题: 我又来这里求牛人：谁会oracle retail etl script?
发信站: BBS 未名空间站 (Thu May 30 10:45:49 2013, 美东)
据说是oracle retail management system里的。
RETL scripts for performing ETL activities in the Oracle Retail Management
System (RMS)
谢谢！ |
|
p**5
发帖数: 2544 | 13 我有个Biostat的Master，也考了SAS base & Advance 证书，在药厂做SAS programmer
的朋友建议我找SAS programmer的工作，但是最近有个做ETL developer的朋友说其实
ETL developer更好，所以想问点情况，找SAS programmer的话容易点，ETL 的话还要
学学datastage，不过我对unix/linux很熟。 |
|
w*r
发帖数: 2421 | 14 ETL programming jobs are dominate by A3 and most of them are very boring
tedious work. In this industry most of those so called ETL architect are
idiots. Since these jobs are most likely binding to specific tool, the
developers are just some tool user. Namely, informatica, data stage,
syncsort, SSIS.
If you want a job earns money, go with ETL development. Plenty of openings
everywhere, but be prepared to deal with idiots all the time and not
necessary more money in the future.
If you want inter... 阅读全帖 |
|
s********m
发帖数: 25 | 15 做database web application developer (Java) 好几年了, 最近有个转去做data
warehouse ETL developer (IBM Netezza, data stage, 的机会, 没有做过ETL, 不知
道哪一个career path更好点? 如果年纪大了的话做data warehouse model/ETL不会更
稳定点? 谢谢 |
|
t****n
发帖数: 10724 | 16 兄弟，别太小看ETL，ETL可以做的很复杂，解决疑难问题，我做过一段时间data
warehouse的ETL. 工作比较少就是了。因为data warehouse只有大公司才需要 |
|
w****y
发帖数: 42 | 17 think you are smart? INFORMATICA is used by many big companies only as the
a low end product intended for mass outsourcing. And if you think it is the
best ETL tool, i wonder how much you
know about ETL. or maybe you just know some Indian contractors who happen to
work inofrmatica at the time.
if ETL is simple, RK wouldn't have to write a separate book about it. |
|
R*****n
发帖数: 355 | 18 【 以下文字转载自 TransferToCS 俱乐部 】
发信人: RyanXin (RyanXin), 信区: TransferToCS
标 题: 【转】Career Talk and ETL Tools
发信站: BBS 未名空间站 (Fri Mar 27 20:12:36 2015, 美东)
Hello All,
Next Tuesday, The career talk will be hosted by James and Derek. The topic "
ETL Tools learning including SSIS, Informatica, Talend" will be discussed
during the session.
You are very welcome to attend this session discussion by using the below
link.
Please register for "Career Talk and ETL Tools Learning" on Mar 31, 2015 8:
30 PM CDT a... 阅读全帖 |
|
c**********6
发帖数: 26 | 19 ETL Informatica Powercenter training is recommended:
while you are hunting job, you may keep learning something. more skills more
opportunities to make ourselves outstanding. There are strong SAS and ETL
informatica powercenter courses. So far, we have not seen others can open
ETL informatica course. But it is hot in California. It is much less
competitive in job market because only a few know it. |
|
R*****n
发帖数: 355 | 20 【 以下文字转载自 TransferToCS 俱乐部 】
发信人: RyanXin (RyanXin), 信区: TransferToCS
标 题: 【转】Career Talk and ETL Tools
发信站: BBS 未名空间站 (Fri Mar 27 20:12:36 2015, 美东)
Hello All,
Next Tuesday, The career talk will be hosted by James and Derek. The topic "
ETL Tools learning including SSIS, Informatica, Talend" will be discussed
during the session.
You are very welcome to attend this session discussion by using the below
link.
Please register for "Career Talk and ETL Tools Learning" on Mar 31, 2015 8:
30 PM CDT a... 阅读全帖 |
|
Y*******n
发帖数: 4646 | 21 请版主保留一会， 谢谢
【 以下文字转载自 Stock 讨论区 】
发信人: YourQueen (Her Highness), 信区: Stock
标 题: 我又来这里求牛人：谁会oracle retail etl script?
发信站: BBS 未名空间站 (Thu May 30 10:45:49 2013, 美东)
据说是oracle retail management system里的。
RETL scripts for performing ETL activities in the Oracle Retail Management
System (RMS)
谢谢！ |
|
t****2
发帖数: 16 | 22 我以前是做Oracle Hyperion consultant ERP ETL SQL 都懂一些
刚刚在Chicago读MBA毕业了
开始想的挺美觉得自己进项目太大没问题，毕竟有一些经验，在四大（PWC）出来的，
上学时候也一直有hr找
毕业想都没想直接跑来湾区，2个月下来各种phone interview 结果都没到F2F就被pass了
1. Team 大部分是老印 面试也是走个形势 根本不招你
2. Hyperion/BI 这块本来就是短期项目(contract居多) 公司大都不愿意support H1
3. 语言能力要求比较高 自己经验技术也还欠缺
兄弟姐妹们 有Hyperion/ERP的职位当然好了，ETL 和 SQL的任何相关的职位我都不排
斥，放心大胆的refer我吧
Entry level的开发（以前做过VBA）也能胜任 ， 滴水之恩当涌泉相报， 欢迎各位多
提高贵意见
或者有华人公司招IT／Finance的 我也可以去
Feel Free to Reach me at : t****[email protected]／408-930-1705
linkedin 欢迎加好友
htt... 阅读全帖 |
|
B***S
发帖数: 73 | 23 Position: ETL QA
Duration: 12 month (will keep renewing)
*Must have ETL, SQL, and Unix experience
Description:
Responsibilities:
· Carry out procedures to ensure that all information systems
products and services meet organization standards and end-user requirements
· Develop and execute test plans and test cases
· Review business requirements and software system designs for
testability
· Perform testing and results analysis and may create test data for
applicatio... 阅读全帖 |
|
g******b
发帖数: 43 | 24 【 以下文字转载自 JobMarket 讨论区 】
发信人: geminibb (爱上queen), 信区: JobMarket
标 题: DC附近ETL/BI developer
关键字: Developer
发信站: BBS 未名空间站 (Thu Jun 11 22:54:18 2015, 美东)
DC郊区utilities公司招Sr. ETL/BI developer.不提供sponsorship。不是求事业发展
的地方,但work life balance好,福利不错。有兴趣请发站内信箱。 |
|
f******1
发帖数: 166 | 25 帮个朋友发的，她因家里有些状况，在藤校读了两年CS就suspend出来工作。现在给一
非IT公司做Project，基本结束了。
目前3+ year SQL server deverloper 经验
2+ year SSIS ETL 经验，结合一些 c#.
SSRS, Tableau, Excel 都用过还不错。
想寻求database developer, ETL developer or Data Analyst Jr level job. 能帮忙
内推的朋友请站内信联系我。
谢谢。 |
|
b**********f
发帖数: 136 | 26 准备电面了，这种ETL的职位也是SDE那种一样的准备么？
平时就是写写SQL，用些tool做ETL啊。用不上Java编程，或者算法，data structure那
些。难道也要准备这些数据结构，算法的东西？ 看网上很多面经，去刷题，难度太大
了。 |
|
c*****0
发帖数: 3 | 27 MS SQL 2008 R2 ). Must have done lot of SSIS, EDW design on SQL Server 2008.
Candidate should live in USA. Work from home opportunity.
Developer - ETL & Data Warehouse
Seeking multiple BI / Data Warehouse Developer / Architects for a global
fortune ten technology client to work on the Microsoft BI Platform. The
candidates need to be dynamic, mature, ambitious, creative, and
autonomous
as the team is Senior Level and experienced in building ETL and data
warehouse. Main... 阅读全帖 |
|
Y*******n
发帖数: 4646 | 28 【 以下文字转载自 Stock 讨论区 】
发信人: YourQueen (Her Highness), 信区: Stock
标 题: 我又来这里求牛人：谁会oracle retail etl script?
发信站: BBS 未名空间站 (Thu May 30 10:45:49 2013, 美东)
据说是oracle retail management system里的。
RETL scripts for performing ETL activities in the Oracle Retail Management
System (RMS)
谢谢！ |
|
u***t
发帖数: 3986 | 29 I am looking for long term (2+ year)resources that's fluent in VBScript, ETL
concept, Writing SQL statements and,
the most important, can send me his/her idea of "the best way to test ETL"
to my mailbox.
Thanks,
d******[email protected] |
|
k*******z
发帖数: 2368 | 30 比如讲，有这么两个数据库，一个是产品，OLTP的。一个是做报告用的，星形。每天晚
上有ETL把产品的新数据发布到星形报告数据库。
问题：产品数据库是24/7，而且数据量很大。ETL要把新数据（过去的24小时产生的）
冗余化，发布到报告数据库上，就要消耗差不多也是24个小时的时间或者更多。如何解
决/改善这种滞后的情况？可能的瓶颈在哪里？ |
|
s**********0
发帖数: 266 | 31 如果一个ETL 处理之前一天数据要24小时或更多， 那这ETL设计绝对有问题。 查查你
的bottleneck在哪？ 我想一般来说光复制OLTP一天的raw data 不应该超过1小时， 剩
下的时间肯定都是花在data processing上了？ |
|
k*****n
发帖数: 361 | 32 被楼上的这么一说我又泄气了，ETL developer工作这么无聊啊，是像admin的工作吗，
我现在做healthcare analyst工作有点意思但是钱少啊，想转ETL啊 |
|
t****n
发帖数: 10724 | 33 原来你做ETL？ 没事做能挣前才是稳拿。大公司有做不完的ETL，实在没事搞几个bug |
|
s**********o
发帖数: 14359 | 34 很多做ETL的很蠢的，当然版上的大牛列外了。
好多连基本的SQL都不会，
对DATA MODEL也一窍不通，我都怀疑他们
能做什么？ETL最牛的还是INFORMATICA，不过呢，
技术太单一的，还是做JAVA的饭碗多。 |
|
p***c
发帖数: 5202 | 35 不矛盾
informatica早就可以java自己写transformation了
好几个开源的ETL都是java的
所以，可以阴阳双修java etl都来 |
|
t****2
发帖数: 16 | 36 我以前是做Oracle Hyperion consultant /ERP/ETL SQL 略懂..
刚刚在Chicago读MBA毕业了
开始想的挺美觉得自己进项目太大没问题，毕竟工作过几年了，上学时候也一直有hr找
毕业想都没想直接跑来湾区，事与愿违了..
1. Team 大部分是老印 面试也是走个形势 根本不招你
2. Hyperion/BI 这块本来就是短期项目(contract居多) 公司大都不愿意support H1
3. 语言能力要求比较高 自己经验技术也还欠缺
有Hyperion/ERP的职位当然好了，ETL 和 SQL的任何相关的职位我都不排斥，放心大胆
的refer我吧
Entry level的开发（以前做过VBA）也能胜任 ， 滴水之恩当涌泉相报， 欢迎各位拍
砖～
t****[email protected]／408-930-1705
linkedin 也欢迎加好友
http://www.linkedin.com/profile/view?id=174122653&goback=%2Enmp |
|
s**********o
发帖数: 14359 | 37 ETL不是弄大规模数据搬迁的么，VPN每秒几百个数据，ETL一两天？ |
|
B*****g
发帖数: 34098 | 38 感谢wyr, goodbug和唐僧, 每人都有包子。
新开一个贴子，原因是技术不拘泥spring, 但还是以java为主，其实就是only java。
不许用M＄，不许用很多DB source。
要求如下：
1. ETL process loading data.
2. Source format storing in DB. (may be also target)
3. Clients can revise their source format through interface. The ETL process
does not need any revise to process the source file in new format.
4. Auto handle performance, like use multi CPU without complicate code.
5. Not too expensive. |
|
p**5
发帖数: 2544 | 39 但是ETL的工作的title 大都是ETL developer，要求的内容和SAS programmer不一样 |
|
c**********6
发帖数: 26 | 40 one option for friends who want to get training and job on BI SAS
Informatica ETL SQL Server SSIS SSRS SSAS Cognos etc. thanks.
渴望求职？转行？进修？
申请BI工作有面试无结果？我们可以你帮助您！
成功只差一步！
祝贺国内医学博士晶女士，住家多年后参加培训已成功获得第一份商业智能好工作！
祝贺美MBA丽女士，住家多年后已成功获得第一份商业智能好工作！
祝贺美化学博士陆先生，已成功转行获得高薪稳定的商业智能BI工作！
USA BI Training Star 面向全美各地；在线商业智能职业培训 (773) 982 - 8168
我们的培训给希望求职转行进修的朋友一个新的选择。
短平快职业培训课程：
• SQL Server (SQL, SSIS, SSRS, SSAS)
• Oracle
• SAS 9.x (based, advanced, workshop/project in bank/ins... 阅读全帖 |
|
M********0
发帖数: 1230 | 41
部分是我的理解 具体FB用什么形式考你在glassdoor上面搜一搜 看看能不能找到相
关的
另外google一下ETL practice会有很多信息 ETL的考法会有很多种
经常用sql说法你处理的都是structured data 对于很多raw data需要clean/extract
这部分可能要多下功夫 |
|
S*******d
发帖数: 190 | 42 CIB - Risk Tech - Oracle Developer Lead- Vice President - Houston-130106134
ETL Developer-140069093
发送简历到c*********[email protected]，我可以提供内退，谢谢！ |
|
n*******h
发帖数: 67 | 43 用scoop2不久完了。干嘛还要写程序ETL? |
|
t***t
发帖数: 6066 | 44 【 以下文字转载自 Database 讨论区 】
发信人: tryst (云淡风轻), 信区: Database
标 题: data warehouse, ETL方面工作和一般software eng比
发信站: BBS 未名空间站 (Sat Mar 1 21:13:22 2014, 美东)
前景和钱景如何？ |
|
S*******d
发帖数: 190 | 45 CIB - Risk Tech - Oracle Developer Lead- Vice President - Houston-130106134
ETL Developer-140069093
发送简历到c*********[email protected]，我可以提供内退，谢谢！ |
|
x*n
发帖数: 33 | 46 谢谢啊！
追问一下。是主要考SQL query，还是也有建table 建trigger 建index的题
另外 我查ETL的wikipage 这个貌似是把hetergeneous的数据 提取 清理 然后放到数据
库或者data warehouse 的过程。如果只考SQL 感觉有点跑题啊。 |
|
g********t
发帖数: 212 | 47 “ 如果SWE可以的话就去当码农了”。。。。不知道这句话是褒义还是贬义... 结合上
下文看来楼主理解是1. etl不是码农2. 且还比码农档次高？
unfortunately...上面两个理解都错了吧
pipeline |
|
S*******i
发帖数: 1529 | 48 汗，没做过ETL也可以面试data scientist了？
pipeline |
|
发帖数: 1 | 49 刚刚发错版了，
一个公司，title一样（即收入相差不大），怎么选？
我自己倾向于SQL Server ( ETL ）的工作 |
|
g*********g
发帖数: 31 | 50 职位是data analyst，工作主要内容都是ETL相关，想问有没有人做过这方面的可以分
享下流程和一些细节吗？非常感谢！ |
|
