基于Hadoop 分布式文件系统的地震勘探大数据样本采集及存储优化
作者:
作者单位:

作者简介:

杨河山(1981—),男,山东德州人,高级工程师,硕士,从事勘探开发大数据技术算法研究与数据处理工作。E-mail:yangheshan751.slyt@sinopec.com。

通讯作者:

基金项目:

中国石化科技攻关项目“老油田开发大数据应用系统集成与示范应用”(P20071-4)。


HDFS-based collection and storage optimization of seismic exploration big data samples
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    随着油气勘探开发智能化应用越来越成熟、应用场景越来越丰富,大规模应用日益临近,样本的分布式存储、 高效采集及并行计算已成为油气勘探开发智能化应用的迫切需求。地震勘探的智能化是油气勘探开发智能化的 重要组成部分。针对地震勘探数据具有的单一文件数据量大、非结构化的特点,在分析地震勘探大数据样本采集需求的基础上,提出基于Hadoop分布式文件系统(HDFS)的大文件分割和合并的解决方案,并对地震勘探数据生成3个不同维度的冗余存储,以提升地震勘探样本的采集效率。测试结果表明,基于HDFS的三倍冗余存储方案在数据量迅速增大的情况下,可以有效地提高地震勘探大数据样本的采集效率,从而满足地震勘探智能化应用需求。

    Abstract:

    As the intelligent application of oil and gas exploration and development matures and application scenarios increase,large-scale application is drawing nearer. As a result,the distributed storage,efficient collection,and parallel com? puting of samples have become urgent requirements of the intelligent application of oil and gas exploration and development. The intelligent application of seismic exploration is an important part of that of oil and gas exploration and development. In view of the large amount of single file data in and the unstructured characteristic of seismic exploration data,this paper analyzes the collection requirements for seismic exploration big data samples,proposes a solution of large file segmentation and merging based on the Hadoop distributed file system(HDFS),and implements redundant storage of seismic exploration data in three dimensions to improve the efficiency of seismic exploration sample collection. The experimental results show that the HDFS-based triple redundant storage solution can effectively improve the efficiency in collecting seismic exploration big data samples under rapid growth in data amount and therefore meet the requirements for intelligent application of seismic exploration.

    参考文献
    相似文献
    引证文献
引用本文

杨河山,张世明,曹小朋,李春雷,姜兴兴.基于Hadoop 分布式文件系统的地震勘探大数据样本采集及存储优化[J].油气地质与采收率,2022,29(1):121~127

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-03-30