without haste but without rest
[HDFS] hdfs 파일을 aws S3로 복사하기 본문
참조 - https://medium.com/dataseries/copy-hadoop-data-hive-to-s3-bucket-d1ffb59279c8
Copy Hadoop Data — Hive to S3 Bucket
WHAT IS S3: S3 stands for “Simple Storage Service” and is offered by Amazon Web Services. It provides a simple to use file object storage…
medium.com
짧게 요약한 아티클
참조 - https://docs.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_admin_distcp_data_cluster_migrate.html
Copying Cluster Data Using DistCp | 5.8.x | Cloudera Documentation
The guidelines mentioned in this section are only applicable for the following sample deployment: Let's assume you have two clusters with the realms: SOURCE and DESTINATION You have data that needs to be copied from SOURCE to DESTINATION Trust exists betwe
docs.cloudera.com
클라우데라 도큐먼트
hadoop distcp -Dfs.s3a.access.key=myAccessKey -Dfs.s3a.secret.key=mySecretKey hdfs://user/hdfs/mydata s3a://myBucket/mydata_backuphdfs 파일 위치는 hue나 hdfs 웹 ui환경에서 복사해서 빠르게 카피카피
'Data Engineering & DataOps > Hadoop' 카테고리의 다른 글
| Hadoop - JAVA_HOME Not Found 이슈 (0) | 2021.09.09 |
|---|---|
| 하둡 권한 문제 (0) | 2021.08.01 |
| HDFS 네임노드 tixd 시작 - We expected txid... 이슈 (0) | 2021.03.02 |
| 하둡 web 권한 문제 - dr.who permission denied (0) | 2021.02.25 |
| 스쿱을 사용해서 MySQL에 저장한 데이터를 HDFS 적재하기 (0) | 2020.07.27 |
Comments