WO2004042930A3 - Content-based segmentation scheme for data compression - Google Patents

Content-based segmentation scheme for data compression Download PDF

Info

Publication number
WO2004042930A3
WO2004042930A3 PCT/US2003/034327 US0334327W WO2004042930A3 WO 2004042930 A3 WO2004042930 A3 WO 2004042930A3 US 0334327 W US0334327 W US 0334327W WO 2004042930 A3 WO2004042930 A3 WO 2004042930A3
Authority
WO
WIPO (PCT)
Prior art keywords
segment
input data
encoded
referenced
references
Prior art date
Application number
PCT/US2003/034327
Other languages
French (fr)
Other versions
WO2004042930A2 (en
Inventor
Steven Mccanne
Michael J Demmer
Original Assignee
Riverbed Technology Inc
Steven Mccanne
Michael J Demmer
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Riverbed Technology Inc, Steven Mccanne, Michael J Demmer filed Critical Riverbed Technology Inc
Priority to DE60328528T priority Critical patent/DE60328528D1/en
Priority to CN2003801049091A priority patent/CN1736030B/en
Priority to EP03781454A priority patent/EP1584139B1/en
Priority to JP2004550188A priority patent/JP4512893B2/en
Priority to AT03781454T priority patent/ATE437477T1/en
Priority to AU2003288962A priority patent/AU2003288962A1/en
Publication of WO2004042930A2 publication Critical patent/WO2004042930A2/en
Publication of WO2004042930A3 publication Critical patent/WO2004042930A3/en
Priority to HK05111175.9A priority patent/HK1076935A1/en

Links

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99941—Database schema or data structure
    • Y10S707/99943—Generating database or data structure, e.g. via user interface
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99951—File or database maintenance
    • Y10S707/99952—Coherency, e.g. same view to multiple users
    • Y10S707/99953—Recoverability

Abstract

In a coding system, input data within a system is encoded. The input data might include sequences of symbols that repeat in the input data or occur in other input data encoded in the system. The encoding includes determining a target segment size, determining a window size, identifying a fingerprint within a window of symbols at an offset in the input data, determining whether the offset is to be designated as a cut point and segmenting the input data as indicated by the set of cut points. For each segment so identified, the encoder determines whether the segment is to be a referenced segment or an unreferenced segment, replacing the segment data of each referenced segment with a reference label and storing a reference binding in a persistent segment store for each referenced segment, if needed. Hierarchically, the process can be repeated by grouping references into groups, replacing the grouped references with a group label, storing a binding between the grouped references and group label, if one is not already present, and repeating the process. The number of levels of hierarchy can be fixed in advanced or it can be determined from the content encoded.
PCT/US2003/034327 2002-10-30 2003-10-28 Content-based segmentation scheme for data compression WO2004042930A2 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
DE60328528T DE60328528D1 (en) 2002-10-30 2003-10-28 E DATA COMPRESSION IN STORAGE AND TRANSMISSION WITH HIERARCHICAL SEGMENT REPRESENTATION
CN2003801049091A CN1736030B (en) 2002-10-30 2003-10-28 Method for encoding input data
EP03781454A EP1584139B1 (en) 2002-10-30 2003-10-28 Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation
JP2004550188A JP4512893B2 (en) 2002-10-30 2003-10-28 A content-based segmentation scheme for data compression during storage and transmission including hierarchical segment representation
AT03781454T ATE437477T1 (en) 2002-10-30 2003-10-28 CONTENT BASED SEGMENTATION SCHEME FOR DATA COMPRESSION DURING STORAGE AND TRANSMISSION WITH HIERARCHICAL SEGMENT REPRESENTATION
AU2003288962A AU2003288962A1 (en) 2002-10-30 2003-10-28 Content-based segmentation scheme for data compression
HK05111175.9A HK1076935A1 (en) 2002-10-30 2005-12-07 Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/285,330 2002-10-30
US10/285,330 US6667700B1 (en) 2002-10-30 2002-10-30 Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation

Publications (2)

Publication Number Publication Date
WO2004042930A2 WO2004042930A2 (en) 2004-05-21
WO2004042930A3 true WO2004042930A3 (en) 2005-08-18

Family

ID=29735736

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/034327 WO2004042930A2 (en) 2002-10-30 2003-10-28 Content-based segmentation scheme for data compression

Country Status (9)

Country Link
US (7) US6667700B1 (en)
EP (2) EP1584139B1 (en)
JP (3) JP4512893B2 (en)
CN (2) CN101599091B (en)
AT (1) ATE437477T1 (en)
AU (1) AU2003288962A1 (en)
DE (1) DE60328528D1 (en)
HK (1) HK1076935A1 (en)
WO (1) WO2004042930A2 (en)

Families Citing this family (217)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7509420B2 (en) * 2000-02-18 2009-03-24 Emc Corporation System and method for intelligent, globally distributed network storage
FR2813743B1 (en) * 2000-09-06 2003-01-03 Claude Seyrat COMPRESSION / DECOMPRESSION PROCESS FOR STRUCTURED DOCUMENTS
US6810398B2 (en) * 2000-11-06 2004-10-26 Avamar Technologies, Inc. System and method for unorchestrated determination of data sequences using sticky byte factoring to determine breakpoints in digital sequences
US7003479B2 (en) * 2000-12-15 2006-02-21 First Data Corporation Systems and methods for ordering and distributing incentive messages
US20020161860A1 (en) * 2001-02-28 2002-10-31 Benjamin Godlin Method and system for differential distributed data file storage, management and access
US6976081B2 (en) * 2002-01-30 2005-12-13 Motorola, Inc. Session initiation protocol compression
US7493911B2 (en) * 2002-08-09 2009-02-24 The Boeing Company Shrouded valve apparatus and related methods
US7650416B2 (en) * 2003-08-12 2010-01-19 Riverbed Technology Content delivery for client-server protocols with user affinities using connection end-point proxies
US6667700B1 (en) * 2002-10-30 2003-12-23 Nbt Technology, Inc. Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation
US7318100B2 (en) * 2003-04-14 2008-01-08 Riverbed Technology, Inc. Cooperative proxy auto-discovery and connection interception
US8176186B2 (en) 2002-10-30 2012-05-08 Riverbed Technology, Inc. Transaction accelerator for client-server communications systems
US7120666B2 (en) * 2002-10-30 2006-10-10 Riverbed Technology, Inc. Transaction accelerator for client-server communication systems
US8364815B2 (en) 2005-03-18 2013-01-29 Riverbed Technology, Inc. Reliability and availability of distributed servers
EP1634162B1 (en) 2003-01-31 2010-07-14 Visto Corporation Asynchronous real-time retrieval of data
WO2004100420A2 (en) * 2003-05-08 2004-11-18 Sap Portals Israel Ltd. A pattern driven message oriented compression apparatus and method
US7296030B2 (en) * 2003-07-17 2007-11-13 At&T Corp. Method and apparatus for windowing in entropy encoding
US8938595B2 (en) * 2003-08-05 2015-01-20 Sepaton, Inc. Emulated storage system
US7464331B2 (en) * 2003-08-18 2008-12-09 Microsoft Corporation System and method for validating hierarchically-organized messages
US8135683B2 (en) * 2003-12-16 2012-03-13 International Business Machines Corporation Method and apparatus for data redundancy elimination at the block level
US7002493B2 (en) * 2004-01-08 2006-02-21 Mathstar, Inc. Boolean logic tree reduction circuit
US20050169223A1 (en) * 2004-01-16 2005-08-04 Crocker Ronald T. Method and apparatus for facilitating a PTT session initiation using an IP-based protocol
US20060047855A1 (en) 2004-05-13 2006-03-02 Microsoft Corporation Efficient chunking algorithm
US7269689B2 (en) * 2004-06-17 2007-09-11 Hewlett-Packard Development Company, L.P. System and method for sharing storage resources between multiple files
US7613787B2 (en) * 2004-09-24 2009-11-03 Microsoft Corporation Efficient algorithm for finding candidate objects for remote differential compression
US20060085740A1 (en) * 2004-10-20 2006-04-20 Microsoft Corporation Parsing hierarchical lists and outlines
US7574409B2 (en) * 2004-11-04 2009-08-11 Vericept Corporation Method, apparatus, and system for clustering and classification
US7849462B2 (en) * 2005-01-07 2010-12-07 Microsoft Corporation Image server
US20070094348A1 (en) * 2005-01-07 2007-04-26 Microsoft Corporation BITS/RDC integration and BITS enhancements
US8073926B2 (en) * 2005-01-07 2011-12-06 Microsoft Corporation Virtual machine image server
US8332485B1 (en) 2005-03-04 2012-12-11 Cisco Technology, Inc. Lock optimization and lock prediction approaches for reducing client-server messages
US20060248194A1 (en) 2005-03-18 2006-11-02 Riverbed Technology, Inc. Connection forwarding
US7098815B1 (en) * 2005-03-25 2006-08-29 Orbital Data Corporation Method and apparatus for efficient compression
US7853962B1 (en) 2005-05-31 2010-12-14 Cisco Technology, Inc. Method and apparatus for optimization of remote procedure call communications
US8521752B2 (en) 2005-06-03 2013-08-27 Osr Open Systems Resources, Inc. Systems and methods for arbitrary data transformations
US8171238B1 (en) 2007-07-05 2012-05-01 Silver Peak Systems, Inc. Identification of data stored in memory
US8095774B1 (en) 2007-07-05 2012-01-10 Silver Peak Systems, Inc. Pre-fetching data into a memory
US8392684B2 (en) 2005-08-12 2013-03-05 Silver Peak Systems, Inc. Data encryption in a network memory architecture for providing data based on local accessibility
US8489562B1 (en) 2007-11-30 2013-07-16 Silver Peak Systems, Inc. Deferred data storage
US8929402B1 (en) 2005-09-29 2015-01-06 Silver Peak Systems, Inc. Systems and methods for compressing packet data by predicting subsequent data
US8811431B2 (en) 2008-11-20 2014-08-19 Silver Peak Systems, Inc. Systems and methods for compressing packet data
US7716307B1 (en) 2005-10-21 2010-05-11 Cisco Technology, Inc. Method and apparatus for reducing client-server messages associated with opening a file
US7844774B2 (en) * 2005-11-16 2010-11-30 Sun Microsystems, Inc. Extensible fingerprinting functions and content addressed storage system using the same
US7636767B2 (en) * 2005-11-29 2009-12-22 Cisco Technology, Inc. Method and apparatus for reducing network traffic over low bandwidth links
US9460060B2 (en) * 2005-12-22 2016-10-04 Xerox Corporation System and method for managing dynamic document references
US8509563B2 (en) * 2006-02-02 2013-08-13 Microsoft Corporation Generation of documents from images
US20070192086A1 (en) * 2006-02-13 2007-08-16 Linfeng Guo Perceptual quality based automatic parameter selection for data compression
US7844652B2 (en) * 2006-04-11 2010-11-30 Emc Corporation Efficient computation of sketches
US7949824B2 (en) * 2006-04-11 2011-05-24 Emc Corporation Efficient data storage using two level delta resemblance
US7562186B2 (en) * 2006-04-11 2009-07-14 Data Domain, Inc. Efficient data storage using resemblance of data segments
US8190742B2 (en) * 2006-04-25 2012-05-29 Hewlett-Packard Development Company, L.P. Distributed differential store with non-distributed objects and compression-enhancing data-object routing
US20070276912A1 (en) * 2006-05-05 2007-11-29 Mike Rybak Apparatus and method for forming and communicating a responsive data message
AU2007247939B2 (en) 2006-05-05 2012-02-09 Hybir Inc. Group based complete and incremental computer file backup system, process and apparatus
US8463843B2 (en) 2006-05-26 2013-06-11 Riverbed Technology, Inc. Throttling of predictive ACKs in an accelerated network communication system
WO2007149687A2 (en) 2006-05-30 2007-12-27 Riverbed Technology, Inc. Selecting proxies from among autodiscovered proxies
US7504969B2 (en) * 2006-07-11 2009-03-17 Data Domain, Inc. Locality-based stream segmentation for data deduplication
US7979584B1 (en) * 2006-07-14 2011-07-12 Emc Corporation Partitioning a data stream using embedded anchors
CN1997011B (en) * 2006-07-26 2011-01-12 白杰 Data partition method and data partition device
US8755381B2 (en) 2006-08-02 2014-06-17 Silver Peak Systems, Inc. Data matching using flow based packet data storage
US8885632B2 (en) 2006-08-02 2014-11-11 Silver Peak Systems, Inc. Communications scheduler
CN1928850B (en) * 2006-08-11 2011-04-13 白杰 Method and apparatus for compressing data based on digital dictionary
US7950062B1 (en) * 2006-08-15 2011-05-24 Trend Micro Incorporated Fingerprinting based entity extraction
US7844581B2 (en) * 2006-12-01 2010-11-30 Nec Laboratories America, Inc. Methods and systems for data management using multiple selection criteria
US7733910B2 (en) * 2006-12-29 2010-06-08 Riverbed Technology, Inc. Data segmentation using shift-varying predicate function fingerprinting
US20080172430A1 (en) * 2007-01-11 2008-07-17 Andrew Thomas Thorstensen Fragmentation Compression Management
US8775663B1 (en) 2007-04-25 2014-07-08 Netapp, Inc. Data replication network traffic compression
EP2153340A4 (en) * 2007-05-08 2015-10-21 Riverbed Technology Inc A hybrid segment-oriented file server and wan accelerator
US7769729B2 (en) 2007-05-21 2010-08-03 Sap Ag Block compression of tables with repeated values
US7836099B2 (en) * 2007-07-31 2010-11-16 Yahoo! Inc. Mail compression scheme with individual message decompressability
US8463759B2 (en) * 2007-09-24 2013-06-11 Ca, Inc. Method and system for compressing data
US8332404B2 (en) * 2007-10-25 2012-12-11 Hewlett-Packard Development Company, L.P. Data processing apparatus and method of processing data
US8099573B2 (en) * 2007-10-25 2012-01-17 Hewlett-Packard Development Company, L.P. Data processing apparatus and method of processing data
DE112007003693B4 (en) * 2007-10-25 2014-05-15 Hewlett-Packard Development Co., L.P. Data processing device and method for data processing
US8150851B2 (en) * 2007-10-25 2012-04-03 Hewlett-Packard Development Company, L.P. Data processing apparatus and method of processing data
WO2009054828A1 (en) * 2007-10-25 2009-04-30 Hewlett-Packard Development Company, L.P. Data processing apparatus and method of processing data
US8140637B2 (en) 2007-10-25 2012-03-20 Hewlett-Packard Development Company, L.P. Communicating chunks between devices
WO2009054834A1 (en) * 2007-10-25 2009-04-30 Hewlett-Packard Development Company, L.P. Data processing apparatus and method of processing data
US20090112861A1 (en) * 2007-10-30 2009-04-30 Honeywell International, Inc. Non-word or non-number search
US8307115B1 (en) 2007-11-30 2012-11-06 Silver Peak Systems, Inc. Network memory mirroring
US9690668B2 (en) * 2008-03-05 2017-06-27 Ca, Inc. Data boundary identification
GB2472520B (en) * 2008-04-25 2012-11-21 Hewlett Packard Development Co Data processing apparatus and method of deduplicating data for data backup
US8620877B2 (en) * 2008-04-30 2013-12-31 International Business Machines Corporation Tunable data fingerprinting for optimizing data deduplication
US8266114B2 (en) 2008-09-22 2012-09-11 Riverbed Technology, Inc. Log structured content addressable deduplicating storage
US8832034B1 (en) 2008-07-03 2014-09-09 Riverbed Technology, Inc. Space-efficient, revision-tolerant data de-duplication
US8743683B1 (en) 2008-07-03 2014-06-03 Silver Peak Systems, Inc. Quality of service using multiple flows
US10164861B2 (en) 2015-12-28 2018-12-25 Silver Peak Systems, Inc. Dynamic monitoring and visualization for network health characteristics
US8370309B1 (en) 2008-07-03 2013-02-05 Infineta Systems, Inc. Revision-tolerant data de-duplication
US10805840B2 (en) 2008-07-03 2020-10-13 Silver Peak Systems, Inc. Data transmission via a virtual wide area network overlay
US9717021B2 (en) 2008-07-03 2017-07-25 Silver Peak Systems, Inc. Virtual network overlay
US8788466B2 (en) * 2008-08-05 2014-07-22 International Business Machines Corporation Efficient transfer of deduplicated data
US8078593B1 (en) 2008-08-28 2011-12-13 Infineta Systems, Inc. Dictionary architecture and methodology for revision-tolerant data de-duplication
US8290915B2 (en) 2008-09-15 2012-10-16 International Business Machines Corporation Retrieval and recovery of data chunks from alternate data stores in a deduplicating system
US8369324B1 (en) * 2008-09-29 2013-02-05 Avaya Inc. Variable compression queue
US8495032B2 (en) 2008-10-01 2013-07-23 International Business Machines Corporation Policy based sharing of redundant data across storage pools in a deduplicating system
US7873060B2 (en) 2008-10-18 2011-01-18 Fortinet, Inc. Accelerating data communication using tunnels
US8117343B2 (en) * 2008-10-28 2012-02-14 Hewlett-Packard Development Company, L.P. Landmark chunking of landmarkless regions
US8751462B2 (en) * 2008-11-14 2014-06-10 Emc Corporation Delta compression after identity deduplication
US8447740B1 (en) 2008-11-14 2013-05-21 Emc Corporation Stream locality delta compression
US8849772B1 (en) * 2008-11-14 2014-09-30 Emc Corporation Data replication with delta compression
WO2010080591A2 (en) * 2008-12-18 2010-07-15 Sumooh Inc. Methods and apparatus for content-aware data partitioning and data de-duplication
US8161255B2 (en) * 2009-01-06 2012-04-17 International Business Machines Corporation Optimized simultaneous storing of data into deduplicated and non-deduplicated storage pools
WO2010090745A1 (en) * 2009-02-06 2010-08-12 Osr Open Systems Resources, Inc. Methods and systems for data storage
US8375182B2 (en) * 2009-02-10 2013-02-12 Hewlett-Packard Development Company, L.P. System and method for segmenting a data stream
US8001273B2 (en) * 2009-03-16 2011-08-16 Hewlett-Packard Development Company, L.P. Parallel processing of input data to locate landmarks for chunks
WO2010108165A1 (en) 2009-03-20 2010-09-23 The Trustees Of Princeton University Systems and methods for network acceleration and efficient indexing for caching file systems
US7979491B2 (en) * 2009-03-27 2011-07-12 Hewlett-Packard Development Company, L.P. Producing chunks from input data using a plurality of processing elements
US20100281077A1 (en) * 2009-04-30 2010-11-04 Mark David Lillibridge Batching requests for accessing differential data stores
US9141621B2 (en) * 2009-04-30 2015-09-22 Hewlett-Packard Development Company, L.P. Copying a differential data store into temporary storage media in response to a request
US8412848B2 (en) * 2009-05-29 2013-04-02 Exagrid Systems, Inc. Method and apparatus for content-aware and adaptive deduplication
US8401181B2 (en) * 2009-06-09 2013-03-19 Emc Corporation Segment deduplication system with encryption of segments
US8762348B2 (en) * 2009-06-09 2014-06-24 Emc Corporation Segment deduplication system with compression of segments
US8731190B2 (en) * 2009-06-09 2014-05-20 Emc Corporation Segment deduplication system with encryption and compression of segments
US9298722B2 (en) 2009-07-16 2016-03-29 Novell, Inc. Optimal sequential (de)compression of digital data
US9438413B2 (en) * 2010-01-08 2016-09-06 Novell, Inc. Generating and merging keys for grouping and differentiating volumes of files
US8200641B2 (en) * 2009-09-11 2012-06-12 Dell Products L.P. Dictionary for data deduplication
US9164969B1 (en) * 2009-09-29 2015-10-20 Cadence Design Systems, Inc. Method and system for implementing a stream reader for EDA tools
US8423520B2 (en) * 2009-11-23 2013-04-16 Dell Products L.P. Methods and apparatus for efficient compression and deduplication
US8633838B2 (en) * 2010-01-15 2014-01-21 Neverfail Group Limited Method and apparatus for compression and network transport of data in support of continuous availability of applications
US8495312B2 (en) * 2010-01-25 2013-07-23 Sepaton, Inc. System and method for identifying locations within data
US8660994B2 (en) * 2010-01-28 2014-02-25 Hewlett-Packard Development Company, L.P. Selective data deduplication
US8797193B2 (en) * 2010-01-29 2014-08-05 Hewlett-Packard Development Company, L.P. Parallel test payload
US8782734B2 (en) * 2010-03-10 2014-07-15 Novell, Inc. Semantic controls on data storage and access
US9292594B2 (en) * 2010-03-10 2016-03-22 Novell, Inc. Harvesting relevancy data, including dynamic relevancy agent based on underlying grouped and differentiated files
WO2011113042A2 (en) * 2010-03-12 2011-09-15 Copiun, Inc. Distributed catalog, data store, and indexing
EP2548122B1 (en) 2010-03-16 2021-06-09 BlackBerry Limited Highly scalable and distributed data de-duplication
US8832103B2 (en) 2010-04-13 2014-09-09 Novell, Inc. Relevancy filter for new data based on underlying files
US8217813B2 (en) * 2010-04-29 2012-07-10 Advanced Micro Devices, Inc. System and method for low-latency data compression/decompression
CN102143039B (en) * 2010-06-29 2013-11-06 华为技术有限公司 Data segmentation method and equipment for data compression
US9141831B2 (en) 2010-07-08 2015-09-22 Texas Instruments Incorporated Scheduler, security context cache, packet processor, and authentication, encryption modules
US9621405B2 (en) 2010-08-24 2017-04-11 Good Technology Holdings Limited Constant access gateway and de-duplicated data cache server
US8682873B2 (en) 2010-12-01 2014-03-25 International Business Machines Corporation Efficient construction of synthetic backups within deduplication storage system
US9798732B2 (en) 2011-01-06 2017-10-24 Micro Focus Software Inc. Semantic associations in data
US20120185612A1 (en) * 2011-01-19 2012-07-19 Exar Corporation Apparatus and method of delta compression
US8688651B2 (en) 2011-01-25 2014-04-01 Sepaton, Inc. Dynamic deduplication
US8732660B2 (en) 2011-02-02 2014-05-20 Novell, Inc. User input auto-completion
US8442986B2 (en) 2011-03-07 2013-05-14 Novell, Inc. Ranking importance of symbols in underlying grouped and differentiated files based on content
US9323769B2 (en) 2011-03-23 2016-04-26 Novell, Inc. Positional relationships between groups of files
WO2012155065A2 (en) * 2011-05-12 2012-11-15 Huawei Technologies, Co., Ltd. Method and system for longest prefix matching of variable-sized hierarchical names by treelets
US8756249B1 (en) * 2011-08-23 2014-06-17 Emc Corporation Method and apparatus for efficiently searching data in a storage system
US8825626B1 (en) 2011-08-23 2014-09-02 Emc Corporation Method and system for detecting unwanted content of files
US9130991B2 (en) 2011-10-14 2015-09-08 Silver Peak Systems, Inc. Processing data packets in performance enhancing proxy (PEP) environment
US8903874B2 (en) 2011-11-03 2014-12-02 Osr Open Systems Resources, Inc. File system directory attribute correction
US9626224B2 (en) 2011-11-03 2017-04-18 Silver Peak Systems, Inc. Optimizing available computing resources within a virtual environment
CN102684827B (en) 2012-03-02 2015-07-29 华为技术有限公司 Data processing method and data processing equipment
JP2013190891A (en) * 2012-03-13 2013-09-26 Hitachi Ltd Data transfer system
US8639672B2 (en) * 2012-03-27 2014-01-28 International Business Machines Corporation Multiplex classification for tabular data compression
US20130279882A1 (en) * 2012-04-23 2013-10-24 Apple Inc. Coding of Video and Audio with Initialization Fragments
CN104718706B (en) * 2012-08-21 2019-11-05 Emc 公司 The method and system of format identification for segmented image data
US9351196B2 (en) 2012-08-31 2016-05-24 International Business Machines Corporation Byte caching in wireless communication networks
US9166620B2 (en) * 2012-09-26 2015-10-20 Qualcomm Incorporated Method and apparatus for a memory based packet compression encoding
US20140095605A1 (en) * 2012-10-01 2014-04-03 Matteo Varvello Method and apparatus for increasing localization of peer-to-peer traffic for content distribution in communication network
WO2014087381A1 (en) 2012-12-07 2014-06-12 Visa International Service Association A token generating component
US9699231B2 (en) * 2012-12-27 2017-07-04 Akamai Technologies, Inc. Stream-based data deduplication using directed cyclic graphs to facilitate on-the-wire compression
US9166619B2 (en) * 2013-01-02 2015-10-20 Verizon Patent And Licensing Inc. Method and system for pattern-based compression
US8704686B1 (en) 2013-01-03 2014-04-22 International Business Machines Corporation High bandwidth compression to encoded data streams
US9306997B2 (en) 2013-01-16 2016-04-05 Cisco Technology, Inc. Method for optimizing WAN traffic with deduplicated storage
US9509736B2 (en) 2013-01-16 2016-11-29 Cisco Technology, Inc. Method for optimizing WAN traffic
US9300748B2 (en) * 2013-01-16 2016-03-29 Cisco Technology, Inc. Method for optimizing WAN traffic with efficient indexing scheme
US9876507B2 (en) 2013-02-22 2018-01-23 Sap Se Semantic compression of structured data
US9887911B2 (en) 2013-02-28 2018-02-06 Xaptum, Inc. Systems, methods, and devices for adaptive communication in a data communication network
US9766832B2 (en) 2013-03-15 2017-09-19 Hitachi Data Systems Corporation Systems and methods of locating redundant data using patterns of matching fingerprints
US9237014B2 (en) * 2013-05-28 2016-01-12 Hong Kong Applied Science & Technology Research Institute Company, Limited Partial CipherText updates using variable-length segments delineated by pattern matching and encrypted by fixed-length blocks
US9256611B2 (en) 2013-06-06 2016-02-09 Sepaton, Inc. System and method for multi-scale navigation of data
CN103609091B (en) * 2013-06-24 2017-01-11 华为技术有限公司 Method and device for data transmission
US9756347B2 (en) * 2013-07-23 2017-09-05 Futurewei Technologies, Inc. Screen content coding systems and methods
US9678973B2 (en) 2013-10-15 2017-06-13 Hitachi Data Systems Corporation Multi-node hybrid deduplication
US9830329B2 (en) 2014-01-15 2017-11-28 W. Anthony Mason Methods and systems for data storage
CN105099460B (en) * 2014-05-07 2018-05-04 瑞昱半导体股份有限公司 Dictionary compression method, dictionary decompression method and dictionary constructing method
JP6318878B2 (en) * 2014-06-04 2018-05-09 富士通株式会社 COMMUNICATION DEVICE, SYSTEM, AND COMMUNICATION PROCESSING METHOD
US9948496B1 (en) 2014-07-30 2018-04-17 Silver Peak Systems, Inc. Determining a transit appliance for data traffic to a software service
DE102015011013B4 (en) 2014-08-22 2023-05-04 Sigma Additive Solutions, Inc. Process for monitoring additive manufacturing processes
US9875344B1 (en) 2014-09-05 2018-01-23 Silver Peak Systems, Inc. Dynamic monitoring and authorization of an optimization device
WO2016081651A1 (en) 2014-11-18 2016-05-26 Sigma Labs, Inc. Multi-sensor quality inference and control for additive manufacturing processes
US9582514B2 (en) * 2014-12-27 2017-02-28 Ascava, Inc. Performing multidimensional search and content-associative retrieval on data that has been losslessly reduced using a prime data sieve
WO2016115284A1 (en) 2015-01-13 2016-07-21 Sigma Labs, Inc. Material qualification system and methodology
KR20160133719A (en) 2015-05-13 2016-11-23 삼성에스디에스 주식회사 Chunk file generating apparatus and method for thereof
US10416915B2 (en) * 2015-05-15 2019-09-17 ScaleFlux Assisting data deduplication through in-memory computation
GB2539966B (en) 2015-07-03 2017-08-30 Sisp Tech Ltd Data processing method and apparatus
TWI534619B (en) * 2015-09-11 2016-05-21 慧榮科技股份有限公司 Methods for logical partitioning dynamically and apparatuses using the same
US10207489B2 (en) 2015-09-30 2019-02-19 Sigma Labs, Inc. Systems and methods for additive manufacturing operations
US10042751B1 (en) * 2015-09-30 2018-08-07 EMC IP Holding Company LLC Method and system for multi-tier all-flash array
US10216748B1 (en) * 2015-09-30 2019-02-26 EMC IP Holding Company LLC Segment index access management in a de-duplication system
US10432484B2 (en) 2016-06-13 2019-10-01 Silver Peak Systems, Inc. Aggregating select network traffic statistics
US10536163B2 (en) 2016-07-04 2020-01-14 Sisp Technologies Ltd. Data processing method and apparatus
US9967056B1 (en) 2016-08-19 2018-05-08 Silver Peak Systems, Inc. Forward packet recovery with constrained overhead
CN106572090A (en) * 2016-10-21 2017-04-19 网宿科技股份有限公司 Data transmission method and system
US10361997B2 (en) 2016-12-29 2019-07-23 Riverbed Technology, Inc. Auto discovery between proxies in an IPv6 network
CN106855930B (en) * 2017-01-04 2019-12-31 成都四方伟业软件股份有限公司 Safe and reliable big data storage system and method
US10771394B2 (en) 2017-02-06 2020-09-08 Silver Peak Systems, Inc. Multi-level learning for classifying traffic flows on a first packet from DNS data
US11044202B2 (en) 2017-02-06 2021-06-22 Silver Peak Systems, Inc. Multi-level learning for predicting and classifying traffic flows from first packet data
US10892978B2 (en) 2017-02-06 2021-01-12 Silver Peak Systems, Inc. Multi-level learning for classifying traffic flows from first packet data
US10257082B2 (en) 2017-02-06 2019-04-09 Silver Peak Systems, Inc. Multi-level learning for classifying traffic flows
US10795860B1 (en) * 2017-04-13 2020-10-06 EMC IP Holding Company LLC WAN optimized micro-service based deduplication
US11171665B2 (en) 2017-09-11 2021-11-09 Nyriad Limited Dictionary-based data compression
US11212210B2 (en) 2017-09-21 2021-12-28 Silver Peak Systems, Inc. Selective route exporting using source type
US20230385242A1 (en) * 2017-10-30 2023-11-30 AtomBeam Technologies Inc. System and methods for bandwidth-efficient data encoding
US11741051B2 (en) * 2017-10-30 2023-08-29 AtomBeam Technologies Inc. System and methods for secure storage for data deduplication
US11397707B2 (en) 2017-10-30 2022-07-26 AtomBeam Technologies Inc. System and method for computer data type identification
US11700013B2 (en) 2017-10-30 2023-07-11 Atombeam Technologies Inc System and method for data compaction and security with extended functionality
US11424760B2 (en) 2017-10-30 2022-08-23 AtomBeam Technologies Inc. System and method for data compaction and security with extended functionality
US10585626B2 (en) 2017-12-07 2020-03-10 International Business Machines Corporation Management of non-universal and universal encoders
US10128868B1 (en) * 2017-12-29 2018-11-13 Intel Corporation Efficient dictionary for lossless compression
US11057352B2 (en) 2018-02-28 2021-07-06 Xaptum, Inc. Communication system and method for machine data routing
US10637721B2 (en) 2018-03-12 2020-04-28 Silver Peak Systems, Inc. Detecting path break conditions while minimizing network overhead
US10965653B2 (en) 2018-03-28 2021-03-30 Xaptum, Inc. Scalable and secure message brokering approach in a communication system
US10805439B2 (en) 2018-04-30 2020-10-13 Xaptum, Inc. Communicating data messages utilizing a proprietary network
US10924593B2 (en) 2018-08-31 2021-02-16 Xaptum, Inc. Virtualization with distributed adaptive message brokering
US10454497B1 (en) 2018-10-22 2019-10-22 Dell Products L. P. Reducing a size of multiple data sets
US10963436B2 (en) * 2018-10-31 2021-03-30 EMC IP Holding Company LLC Deduplicating data at sub-block granularity
US10938877B2 (en) 2018-11-30 2021-03-02 Xaptum, Inc. Optimizing data transmission parameters of a proprietary network
US10912053B2 (en) 2019-01-31 2021-02-02 Xaptum, Inc. Enforcing geographic restrictions for multitenant overlay networks
US11153119B2 (en) 2019-10-15 2021-10-19 Cisco Technology, Inc. Dynamic discovery of peer network devices across a wide area network
CN110830361B (en) * 2019-10-22 2021-12-07 新华三信息安全技术有限公司 Mail data storage method and device
CA3093681C (en) * 2019-11-26 2021-11-09 Citrix Systems, Inc. Document storage and management
US11366735B2 (en) 2020-08-20 2022-06-21 Bank Of America Corporation Dynamic data storage management
CN114328373A (en) * 2020-09-29 2022-04-12 伊姆西Ip控股有限责任公司 Method, electronic device and computer program product for managing a file system
US11681659B2 (en) * 2021-05-21 2023-06-20 Red Hat, Inc. Hybrid file compression model
US11687266B2 (en) 2021-05-26 2023-06-27 Red Hat, Inc. Managing deduplication operations based on a likelihood of duplicability

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6341171B1 (en) * 1995-01-10 2002-01-22 De Lanauze Pierre Method and apparatus for scanning and storing a low resolution optical representation of a finger's image

Family Cites Families (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4377803A (en) * 1980-07-02 1983-03-22 International Business Machines Corporation Algorithm for the segmentation of printed fixed pitch documents
US4641274A (en) 1982-12-03 1987-02-03 International Business Machines Corporation Method for communicating changes made to text form a text processor to a remote host
US5414850A (en) * 1991-08-23 1995-05-09 Stac Electronics, Inc. System for transparently compressing data files in a computer system
US6076084A (en) 1994-01-03 2000-06-13 Norton-Lambert Corp. File transfer method and apparatus utilizing delimiters
DE4422648C2 (en) 1994-06-28 1998-09-03 Rene P Schmid Sollriß joint rail
US5822746A (en) * 1994-07-05 1998-10-13 Trustus Pty Ltd Method for mapping a file specification to a sequence of actions
US5737594A (en) * 1994-07-05 1998-04-07 Trustus Pty Ltd. Method for matching elements of two groups
US5561421A (en) * 1994-07-28 1996-10-01 International Business Machines Corporation Access method data compression with system-built generic dictionaries
EP0700229B1 (en) * 1994-08-22 2006-06-28 Fujitsu Limited Connectionless communications system, test method, and intra-station control system
US5990810A (en) * 1995-02-17 1999-11-23 Williams; Ross Neil Method for partitioning a block of data into subblocks and for storing and communcating such subblocks
EP0826181A4 (en) * 1995-04-11 2005-02-09 Kinetech Inc Identifying data in a data processing system
US5754774A (en) 1996-02-15 1998-05-19 International Business Machine Corp. Client/server communication system
US6791947B2 (en) * 1996-12-16 2004-09-14 Juniper Networks In-line packet processing
US6075876A (en) 1997-05-07 2000-06-13 Draganoff; Georgi Hristoff Sliding yardsticks fingerprint enrollment and verification system and method
US6301378B1 (en) * 1997-06-03 2001-10-09 R2 Technology, Inc. Method and apparatus for automated detection of masses in digital mammograms
US6148342A (en) * 1998-01-27 2000-11-14 Ho; Andrew P. Secure database management system for confidential records using separately encrypted identifier and access request
US6415329B1 (en) * 1998-03-06 2002-07-02 Massachusetts Institute Of Technology Method and apparatus for improving efficiency of TCP/IP protocol over high delay-bandwidth network
US6452915B1 (en) * 1998-07-10 2002-09-17 Malibu Networks, Inc. IP-flow classification in a wireless point to multi-point (PTMP) transmission system
US6163811A (en) * 1998-10-21 2000-12-19 Wildseed, Limited Token based source file compression/decompression and its application
US6178461B1 (en) 1998-12-08 2001-01-23 Lucent Technologies Inc. Cache-based compaction technique for internet browsing using similar objects in client cache as reference objects
US7409546B2 (en) * 1999-10-20 2008-08-05 Tivo Inc. Cryptographically signed filesystem
CN1174374C (en) * 1999-06-30 2004-11-03 国际商业机器公司 Method and device for parallelly having speech recognition, classification and segmentation of speaker
US6915302B1 (en) * 1999-10-01 2005-07-05 International Business Machines Corporation Method, system, and program for accessing files in a file system
US6449658B1 (en) * 1999-11-18 2002-09-10 Quikcat.Com, Inc. Method and apparatus for accelerating data through communication networks
DE19962278A1 (en) * 1999-12-23 2001-08-02 Heidenhain Gmbh Dr Johannes Position measuring device
US6553141B1 (en) * 2000-01-21 2003-04-22 Stentor, Inc. Methods and apparatus for compression of transform data
US7412462B2 (en) * 2000-02-18 2008-08-12 Burnside Acquisition, Llc Data repository and method for promoting network storage of data
US6704730B2 (en) 2000-02-18 2004-03-09 Avamar Technologies, Inc. Hash file system and method for use in a commonality factoring system
WO2001061563A1 (en) * 2000-02-18 2001-08-23 Avamar Technologies, Inc. Hash file system and method for use in a commonality factoring system
JP2002032236A (en) * 2000-07-13 2002-01-31 Nec Commun Syst Ltd Data communication equipment and its data communication method
US6856651B2 (en) * 2000-07-25 2005-02-15 Peribit Networks, Inc. System and method for incremental and continuous data compression
US7047281B1 (en) * 2000-08-08 2006-05-16 Fineground Networks Method and system for accelerating the delivery of content in a networked environment
US6883035B2 (en) 2000-11-16 2005-04-19 Telefonaktiebolaget Lm Ericsson (Publ) System and method for communicating with temporary compression tables
WO2002056478A1 (en) * 2001-01-11 2002-07-18 Koninklijke Philips Electronics N.V. Data compression method with identifier of regressive string reference
US7310687B2 (en) * 2001-03-23 2007-12-18 Cisco Technology, Inc. Methods and systems for managing class-based condensation
US7159014B2 (en) * 2001-06-04 2007-01-02 Fineground Networks Method and system for efficient and automated version management of embedded objects in web documents
US7370120B2 (en) 2001-12-07 2008-05-06 Propel Software Corporation Method and system for reducing network latency in data communication
US7143191B2 (en) * 2002-06-17 2006-11-28 Lucent Technologies Inc. Protocol message compression in a wireless communications system
US6678828B1 (en) * 2002-07-22 2004-01-13 Vormetric, Inc. Secure network file access control system
US6667700B1 (en) 2002-10-30 2003-12-23 Nbt Technology, Inc. Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation
US7269689B2 (en) * 2004-06-17 2007-09-11 Hewlett-Packard Development Company, L.P. System and method for sharing storage resources between multiple files
US7366859B2 (en) * 2005-10-06 2008-04-29 Acronis Inc. Fast incremental backup method and system
CN101326492B (en) * 2005-10-06 2013-07-17 雷德本德有限公司 Methods and systems for updating content including a compressed version
US20070150887A1 (en) * 2005-12-22 2007-06-28 Shapiro Alan J Apparatus and method for selectively dispensing soft assets

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6341171B1 (en) * 1995-01-10 2002-01-22 De Lanauze Pierre Method and apparatus for scanning and storing a low resolution optical representation of a finger's image

Also Published As

Publication number Publication date
US20040174276A1 (en) 2004-09-09
JP4533454B2 (en) 2010-09-01
US20110267207A1 (en) 2011-11-03
HK1076935A1 (en) 2006-01-27
AU2003288962A1 (en) 2004-06-07
US20070018858A1 (en) 2007-01-25
JP2006505217A (en) 2006-02-09
EP1584139A4 (en) 2006-04-26
CN1736030B (en) 2010-07-14
EP1584139A2 (en) 2005-10-12
US6961009B2 (en) 2005-11-01
US6667700B1 (en) 2003-12-23
US20050162288A1 (en) 2005-07-28
AU2003288962A8 (en) 2004-06-07
CN101599091A (en) 2009-12-09
US7477166B2 (en) 2009-01-13
WO2004042930A2 (en) 2004-05-21
JP2009171636A (en) 2009-07-30
JP4512893B2 (en) 2010-07-28
EP1584139B1 (en) 2009-07-22
JP2009171635A (en) 2009-07-30
US7116249B2 (en) 2006-10-03
US7852237B2 (en) 2010-12-14
US8508389B2 (en) 2013-08-13
DE60328528D1 (en) 2009-09-03
CN1736030A (en) 2006-02-15
US6828925B2 (en) 2004-12-07
US20090079597A1 (en) 2009-03-26
CN101599091B (en) 2012-11-28
EP2093885A1 (en) 2009-08-26
US20060061495A1 (en) 2006-03-23
ATE437477T1 (en) 2009-08-15

Similar Documents

Publication Publication Date Title
WO2004042930A3 (en) Content-based segmentation scheme for data compression
US8838551B2 (en) Multi-level database compression
WO2007149358A1 (en) Data compression
TW200815980A (en) Method and system for optimizing storage
CN101807208A (en) Method for quickly retrieving video fingerprints
CN112953550B (en) Data compression method, electronic device and storage medium
CN103067022A (en) Nondestructive compressing method, uncompressing method, compressing device and uncompressing device for integer data
CN1924925A (en) Document data waterprint embedded method
CN105427577A (en) Infrared data processing method, processing device and decoding method
CN109598334B (en) Sample generation method and device
WO2004015629A3 (en) Identification of digital data sequences
CN1758756A (en) Method and device for binary coding wait coding data
CN101615911B (en) Coding and decoding methods and devices
CN111144117A (en) Knowledge graph Chinese address disambiguation method
CN102650969A (en) Method and device for obtaining and updating context probability model value of bins
JP5913748B2 (en) Secure and lossless data compression
CN1294759C (en) Video code method
US20090307243A1 (en) Coding Positions of Data Elements In A Data Structure
WO2005006562A1 (en) A method of decoding variable length prefix codes
CN1187731A (en) Method and apparatus for encoding contour image of object in video signal
CN105117403A (en) Log data fragmentation and query method and apparatus
CN115563058A (en) Similar case retrieval method based on element extraction
CN105320685A (en) Hash coding method and device
CN108092670A (en) Coding method and device
CN104809747A (en) Statistical method of image histogram and system thereof

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 168273

Country of ref document: IL

WWE Wipo information: entry into national phase

Ref document number: 2004550188

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2003781454

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 1053/CHENP/2005

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 20038A49091

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2003781454

Country of ref document: EP