CN100452739C - Optical parallel transmission in usage of grid - Google Patents
Optical parallel transmission in usage of grid Download PDFInfo
- Publication number
- CN100452739C CN100452739C CNB2006100784851A CN200610078485A CN100452739C CN 100452739 C CN100452739 C CN 100452739C CN B2006100784851 A CNB2006100784851 A CN B2006100784851A CN 200610078485 A CN200610078485 A CN 200610078485A CN 100452739 C CN100452739 C CN 100452739C
- Authority
- CN
- China
- Prior art keywords
- grid
- optical
- node
- gridding resource
- gridding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
This invention discloses an optical parallel transmission method which is open to grid application. When grid users send grid operations to grid server; the server finds the operational grid resources according to grid resource list. According to this grid resource node address inquiry optical network, find the spare available optical network resource; based on the above inquiry result, send one or more optical channel-building requests to optical control layer on demand; after the optical control layer establishes the corresponding optical channel, it returns to the result and updates the list of optical network resource; grid server begin to operate the user submit grid operation; after the operation is finished, it informs optical control layer to close the optical channel links. This invention can solve the capacity demand problem that grid application large bandwidth data transmission, improve the data transmission speed and make full use of the grid network bandwidth resources.
Description
Technical field
The present invention relates to a kind of method that realizes the light parallel transmission, relate in particular to a kind of light parallel transmission method, belong to the optical communication technique field towards grid application.
Background technology
Along with the fast development of national economy, national defense construction and high-tech, growing field need solve the calculating of extensive scientific and engineering technology.In order to solve the actual demand of people to high-performance calculation, people have proposed concept of grid computing in conjunction with the development of Internet technology.
Grid is to use for reference a notion of coming from power grid, and the expression computing power can be as electric power with computational resource, and these services that whom are, how provide of being concerned about are not provided " just turn on the power switch and can use ".The target of grid is abstract and quantizes computational resource, can finish the relevant work of calculating of " quantitatively " whenever and wherever possible by network " by norm ".By adopting grid to coordinate the computational resource and the storage resources of allotment far-end, can realize the fast processing of task.
Along with the raising greatly of bandwidth and network speed, new variation has taken place in the mode of Distributed Calculation, and grid has become the developing direction of Next Generation Internet.But the physical transfer environment of grid application mainly still adopts traditional interconnected network mode at present, and promptly grid node links to each other by the Internet, and packet is transmitted by router stores, and router links to each other by fixing optical channel.Under existing internet environment, the grid data transmission mechanism is as follows:
Step 1: grid user is submitted grid work to grid service server;
Step 2: server is sought idle computational resource and the storage resources of carrying out operation according to the distribution situation of computational resource and storage resources;
Step 3: data are mail on the computational resource and storage resources that chooses by high speed router;
Step 4: high speed router sends to Optical Transmission Network OTN by the light mouth with data;
Step 5: Optical Transmission Network OTN connects transfer of data to next high speed router by fixing light;
Step 6: next high speed router repeating step 4 and 5, up to data being sent to the target gridding resource;
Step 7: gridding resource is carried out grid work after obtaining data, with the operation execution result by returning to server (transmitting step is identical with step 4-6);
Step 8: grid service server returns result of calculation to grid user.
Above-mentioned grid physical transfer environment based on the Internet can not fully satisfy big capacity and the distributed business demand that extensive grid service is brought.For this reason, people have carried out number of research projects, think that using optical fiber communication and dense wave division multipurpose DWDM facility in grid network is a promising technical solution.
But existing WDM technology has only solved the big volume transport problem of point-to-point.Because the distributed characteristics of grid application, the applicant of mesh services and supplier are positioned on the dispersion node of three or more probably, in order to allow these nodes can collaborative work, also need to provide parallel fast the connection to create a mechanism, but this parallel connection mechanism does not still have the mature technique scheme at present.In recent years, owing to introduce ASON (ASON) technology, the user can be by uni interface to the dynamic bandwidth application resource of optical-fiber network, and still present this application still is confined to the mode connected in series of point-to-point.
In addition, because the transmission capacity of simple optical fiber is more much bigger than the core capacity of Internet, but the disposal ability of existing router can't be mated growing light transmission speed at all, causes the waste of a large amount of light transmission capacity.If continue to adopt existing serial transmission mode, a lot of data-intensive grid application all can't be responded fast, can't make full use of the bandwidth resources of optical transmission.
Summary of the invention
The objective of the invention is at the demand of grid application, a kind of light parallel transmission method towards grid application is provided big capacity and distributed Network Transmission.This method adopts the GMPLS protocol groups to realize photosphere control based on the Gridftp of the GT4.0 of Globus tissue issue.
For realizing above-mentioned goal of the invention, the present invention adopts following technical scheme:
A kind of light parallel transmission method towards grid application is characterized in that may further comprise the steps:
Step 1, the grid application node is submitted the grid work request to grid service server;
Step 2, the tabulation of grid service server inquiry gridding resource, the gridding resource node of job requirements is satisfied in discovery, obtains the network address of this gridding resource node, continues execution in step 3; If do not satisfy the gridding resource node of job requirements, then return results informs that the user does not have the available light gridding resource at present, and service finishes;
Step 3, the inquiry optical network resource is tabulated, and finds the idle available light network resource information of this gridding resource node, continues execution in step 4; If this gridding resource node does not all have idle available light Internet resources, inform that then the user does not have idle available light Internet resources at present, service finishes;
Step 4, according to Query Result, this grid service server is initiated one or more of optical channels to optical control layer as required and is set up request, and this optical channel is to the passage the gridding resource node with idle available light Internet resources from this grid application node;
After step 5, optical control layer are set up the corresponding light passage, return results;
Step 6, grid service server begin to carry out the grid work request that the user submits to;
Step 7, operation is finished, and notifies this optical control layer to close this optical channel and connects.
Wherein, in the described step 3, described inquiry optical network resource tabulation, the step that finds the idle available light network resource information of this gridding resource node comprises:
There is being a gridding resource node to satisfy under the situation of required gridding resource,, judging between this grid application node and this gridding resource node and can set up N bar optical channel that N is the integer greater than 1 according to the optical network resource tabulation situation of this gridding resource node; And the size of data of Chuan Songing as required, judgement need be set up maximum M bar optical channels; If M 〉=N promptly sets up N bar optical channel; If M＜N promptly sets up M bar optical channel.
Possess under the situation of required gridding resource having more than a gridding resource node, optical network resource tabulation situation according to each gridding resource node, if the available light Internet resources are arranged, then set up 1 optical channel respectively between each gridding resource node at this grid application node more than 1 gridding resource node; If have only 1 gridding resource node to have 1 optical channel resource, then set up 1 optical channel between this gridding resource node at this grid application node; If have only 1 gridding resource node to have the available light Internet resources, and this gridding resource node can be set up greater than 1 optical channel, then judges to set up N bar optical channel between this grid application node and this gridding resource node, N is the integer greater than 1; And the size of data of Chuan Songing as required, judgement need be set up maximum M bar optical channels; If M 〉=N promptly sets up N bar optical channel; If M＜N promptly sets up M bar optical channel.
In the described step 6, when initiating the data transport service request according to the Gridftp agreement, wherein the Gridftp control connection is set up a TCP connection by the Internet, and the data connection is based upon optical control layer.
Each optical channel connects Gridftp program of startup.
Light parallel transmission method towards grid application of the present invention has the following advantages and effect: solved the capacity requirement of big bandwidth data transmission in the grid application; Improved the transmission speed of data; The bandwidth resources of grid network have been made full use of.
Description of drawings
The present invention is further illustrated below in conjunction with the drawings and specific embodiments.
Fig. 1 is a photon grid network structural representation used in the present invention;
Fig. 2 is the photon grid node data transmission process flow chart that embodies the method for the invention;
Fig. 3 is that parallel optical channel is set up the process chart of process in the described data transmission procedure of Fig. 4.
Fig. 4 is an example of the light parallel transmission passage realized in this photon grid network.
Fig. 5 is another example of the light parallel transmission passage realized in this photon grid network.
Embodiment
In the embodiment shown in fig. 1, provided the structural representation of a grid network used in the present invention.As shown in Figure 1, n grid node arranged, pass through optical network communication between grid node.The PC of each node has 2 optical interfaces, 2 common network interface cards.Common network interface card is used for setting up the Gridftp control connection, and optical interface is used as the interface equipment of setting up the optical channel that transmits data.
In the present invention, said gridding resource is meant the traditional gridding resource except that Internet resources, as: computational resource, storage resources, data, application program, all kinds of e-files or the like.And Internet resources are meant each gridding resource node of connection, for the facility of transfer of data and communication port is provided between node, can be optical cross-connect, optical exchanger spare, optical fiber, wavelength, wave band, universal tag, light time crack and interface etc.
A distinguishing feature of the present invention is based on the Gridftp of the GT4.0 of Globus (http://www.globus.org/) tissue issue, adopts the GMPLS protocol groups to realize photosphere control.The FTP of standard sets up two TCP connections between Client and Server traditionally: control connection is connected with data, and control connection is used for sending order and receives response; Data connect and are used for transfer of data.In the present invention, control connection remains by the Internet and sets up a TCP connection, and data connect then sets up optical channel to the optical control layer request, has several optical channels just to start several Gridftp programs simultaneously.
Gridftp is a kind of safe and reliable high-performance data host-host protocol, and it is based on this popular the Internet file transportation protocol of FTP, and at the high bandwidth wide area network it is optimized.This agreement can be supported following function, thereby can realize a kind of robust transmission mechanism:
Parallel data transmission: use a plurality of TCP streams than using single TCP stream to improve bandwidth.The parallel data transmission is provided support by FTP order expansion and data channel expansion.
(Grid Security Infrastructure is GSI) with the kerberos authentication support: control various data integrities and other setting of confidentiality level by the user for grid security infrastructure.This function provides healthy and strong and authentication flexibly, integrality and confidentiality mechanism for transmitting file.
Third party's control of transfer of data: support to be large-scale distributed community management large data collection.It can control the third party to the transmission between the storage server.
Block data transmission: data can be cut apart and be placed on a plurality of servers, assemble bandwidth thereby improve.Gridftp supports the block data transmission by the expansion that is defined in grid forum (Grid Forum) draft.
Partial document transmits: to require application program to transmit whole file different with standard FTP, some zone that novel FTP order supports to transmit file.
Reliable transfer of data: fault recovery method can be handled transient network fault and server failure, can restart the transmission of failure simultaneously.
Hand-guided TCP buffer size: support to obtain maximum TCP/IP bandwidth.
Integrated detection (instrumentation): support to return to restart and performance indicia.
About further specifying of Gridftp, can be with reference to its white paper (network address be http://www-fp.globus.org/datagrid/deliverables/C2WPdraft3.pdf).Just do not describe in detail one by one at this.
With reference to shown in Figure 2, when grid user after grid service server is submitted grid work to, the execution in step of operation is:
Step 1: grid user is submitted grid work to grid service server.
Step 2: the tabulation of grid service server inquiry gridding resource, the gridding resource node of job requirements is satisfied in discovery, obtains this node address.
When step 2 was found gridding resource, the available grids resource that meets the grid application demand of discovery had one or more.In addition, when searching gridding resource, except the node address that obtains the gridding resource place, also can know simultaneously some association attributeses of available resources, as resource size etc.These attribute informations will obtain using in follow-up light parallel transmission.
After finding the available grids resource, the resource node address lookup optical network resource tabulation according to above-mentioned acquisition finds out the optical network resource that node corresponding possesses, and as shown in Figure 3, at this moment two kinds of situations is arranged:
1) only a grid node possesses under the required gridding resource situation, according to optical network resource tabulation situation, if between grid application node and resource node, can set up N (N＞1) bar optical channel, the size of data that transmits at first as required then, judgement need be set up maximum M bar optical channels; At this moment according to M, N size, carry out following steps:
If A. M 〉=N promptly sets up N bar optical channel;
If B. M＜N promptly sets up M bar optical channel.
2) there is the individual grid node of P (P＞1) to possess required gridding resource.According to optical network resource tabulation situation, only set up 1 optical channel respectively between P resource node at the grid application node, if the grid application node is to no available light passage between P-1 resource node, and only more than 1 optical channel is arranged, then follows 1 to 1 resource node wherein) in regulation.
Step 3: the inquiry optical network resource is tabulated, and finds the idle available light network resource information of gridding resource node.
Step 4:, initiate one or more of optical channels to optical control layer as required and set up request according to Query Result.
Step 5: after optical control layer was set up the corresponding light passage, return results was given grid middleware, upgraded the optical network resource tabulation simultaneously.
The example of the optical channel that step 5 is set up such as Fig. 4 or shown in Figure 5.
Step 6: grid service server begins to carry out the grid work that the user submits to.
Step 7: operation is finished, and the notice optical control layer is closed optical channel and connected, and upgrades the optical network resource tabulation simultaneously immediately.
In the present invention, Internet resources Once you begin use, and exclusively enjoy for grid work.The sharing of Internet resources has significantly different in these characteristics and the traditional grid computing.
Submit the grid work request when grid user to grid service server, grid node transmits data if desired, then will be by following flow processing:
Step 1: the grid node middleware is according to the grid work request, and satisfactory resource situation is searched in the tabulation of inquiry gridding resource.
Step 2: if do not find satisfactory available grids resource, promptly return results informs that the user does not have the available grids resource at present, and service finishes; If found available resource, then according to gridding resource place node IP address, the tabulation of inquiry optical network resource;
Step 3: if Query Result is no available optical network resource, inform that then the user does not have the available light Internet resources at present, service finishes; If idle available optical network resource is arranged, then sets up suitable optical channel according to the handling process of Fig. 3;
Step 4: after optical control layer is set up the corresponding light passage, return results;
Step 5: grid service server begins to carry out the grid work request that the user submits to, begins to transmit data, upgrades the optical network resource tabulation simultaneously;
Step 6: operation is finished, and the notice optical control layer is closed optical channel and connected;
Step 7: upgrade the optical network resource tabulation once more.
In the step 3 of above-mentioned photon grid node data transmission process flow process, as free time during available optical network resource, the step of setting up suitable parallel optical channel is as follows:
Step 31: judge whether to possess the gridding resource node of required gridding resource, if do not have then return results; If have, then carry out next step;
Step 32: judge the idle available light passage of gridding resource node, if having the available light passage or have only a gridding resource node to have 1 available light passage more than 1 gridding resource node, then execution in step 33; If have only a gridding resource node to have the available light passage, and the available light passage is more than 1, and then execution in step 34;
Step 33: set up 1 optical channel to each gridding resource node;
Step 34: relatively according to optical network resource tabulation situation, the optical channel quantity N that between grid application node and resource node, can set up, maximum optical channel quantity that definite needs are set up with the size of data that transmits as required are M; If M 〉=N then sets up N bar optical channel; If M＜N then sets up M bar optical channel.
In the present invention, Data Transport Protocol uses Gridftp, connects for each optical channel to start a Gridftp program, and Gridftp is also followed in cutting apart of file data.Wherein the GridFTP control connection is still set up a TCP connection by the Internet, and data connect then sets up optical channel to the optical control layer request.There are several optical channels just to start several Gridftp programs simultaneously.And grid middleware uses Globus toolkit4.0, with respect to the sharing of Internet resources in traditional grid computing, exclusively enjoys for grid work when the Internet resources here use.On the other hand, existing ASON adopts general multiple protocols label exchange (GMPLS) as the control signaling protocol on control plane, connects and the controlling links function to realize dynamic optical channel.The present invention also adopts the basis of this control signaling protocol as photosphere control.How using the GMPLS agreement is all very familiar routine techniques of those skilled in the art of optical communication field, has not just given unnecessary details at this.
Above the light parallel transmission method towards grid application of the present invention is had been described in detail, but specific implementation form of the present invention is not limited thereto.For the those skilled in the art in present technique field, the various conspicuous change of under the situation of spirit that does not deviate from the method for the invention and claim scope it being carried out is all within protection scope of the present invention.
Claims (5)
1. light parallel transmission method towards grid application is characterized in that may further comprise the steps:
Step 1, the grid application node is submitted the grid work request to grid service server;
Step 2, the tabulation of grid service server inquiry gridding resource, the gridding resource node of job requirements is satisfied in discovery, obtains the network address of this gridding resource node, continues execution in step 3; If do not satisfy the gridding resource node of job requirements, then return results informs that the user does not have the available light gridding resource at present, and service finishes;
Step 3, the inquiry optical network resource is tabulated, and finds the idle available light network resource information of this gridding resource node, continues execution in step 4; If this gridding resource node does not all have idle available light Internet resources, inform that then the user does not have idle available light Internet resources at present, service finishes;
Step 4, according to Query Result, this grid service server is initiated one or more of optical channels to optical control layer as required and is set up request, and this optical channel is to the passage the gridding resource node with idle available light Internet resources from this grid application node;
After step 5, optical control layer are set up the corresponding light passage, return results;
Step 6, grid service server begin to carry out the grid work request that the user submits to;
Step 7, operation is finished, and notifies this optical control layer to close this optical channel and connects.
2. smooth parallel transmission method as claimed in claim 1 is characterized in that:
In the described step 3, described inquiry optical network resource tabulation, the step that finds the idle available light network resource information of this gridding resource node comprises:
There is being a gridding resource node to satisfy under the situation of required gridding resource,, judging between this grid application node and this gridding resource node and can set up N bar optical channel that N is the integer greater than 1 according to the optical network resource tabulation situation of this gridding resource node; And the size of data of Chuan Songing as required, judgement need be set up maximum M bar optical channels; If M 〉=N promptly sets up N bar optical channel; If M＜N promptly sets up M bar optical channel.
3. smooth parallel transmission method as claimed in claim 1 is characterized in that:
In the described step 3, described inquiry optical network resource tabulation, the step that finds the idle available light network resource information of this gridding resource node further comprises:
Possess under the situation of required gridding resource having more than a gridding resource node, optical network resource tabulation situation according to each gridding resource node, if the available light Internet resources are arranged, then set up 1 optical channel respectively between each gridding resource node at this grid application node more than 1 gridding resource node; If have only 1 gridding resource node to have 1 optical channel resource, then set up 1 optical channel between this gridding resource node at this grid application node; If have only 1 gridding resource node to have the available light Internet resources, and this gridding resource node can be set up greater than 1 optical channel, then judge between this grid application node and this gridding resource node and can set up N bar optical channel, and the size of data of Chuan Songing as required, judgement need be set up maximum M bar optical channels; If M 〉=N promptly sets up N bar optical channel; If M＜N promptly sets up M bar optical channel.
4. smooth parallel transmission method as claimed in claim 1 is characterized in that:
In the described step 6, when initiating the data transport service request according to the Gridftp agreement, wherein the Gridftp control connection is set up a TCP connection by the Internet, and the data connection is based upon optical control layer.
5. smooth parallel transmission method as claimed in claim 4 is characterized in that:
In the described step 6, each optical channel connects Gridftp program of startup.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100784851A CN100452739C (en) | 2006-05-30 | 2006-05-30 | Optical parallel transmission in usage of grid |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100784851A CN100452739C (en) | 2006-05-30 | 2006-05-30 | Optical parallel transmission in usage of grid |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1984006A CN1984006A (en) | 2007-06-20 |
| CN100452739C true CN100452739C (en) | 2009-01-14 |
Family
ID=38166258
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100784851A Expired - Fee Related CN100452739C (en) | 2006-05-30 | 2006-05-30 | Optical parallel transmission in usage of grid |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100452739C (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101227500B (en) * | 2008-02-21 | 2010-07-21 | 上海交通大学 | Task scheduling method based on optical grid |
| CN101621530B (en) * | 2008-07-04 | 2012-05-30 | 北京邮电大学 | Method and device for dispatching load balancing network resource based on light path share |
| WO2020199143A1 (en) * | 2019-04-03 | 2020-10-08 | 华为技术有限公司 | Ai training network and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5875415A (en) * | 1995-05-12 | 1999-02-23 | Symbol Technologies, Inc. | Universal host interface for data acquisition systems |
| US6488209B1 (en) * | 1999-01-29 | 2002-12-03 | Intermec Ip Corp. | Automatic data collection device that dynamically wedges data transmitted to data consumers |
| CN1540481A (en) * | 2003-10-31 | 2004-10-27 | 清华大学 | Method for using synergic multiple computer engines based on gridding environment |
| CN1747368A (en) * | 2004-09-10 | 2006-03-15 | 华为技术有限公司 | Restoring method of grid network service |
-
2006
- 2006-05-30 CN CNB2006100784851A patent/CN100452739C/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5875415A (en) * | 1995-05-12 | 1999-02-23 | Symbol Technologies, Inc. | Universal host interface for data acquisition systems |
| US6488209B1 (en) * | 1999-01-29 | 2002-12-03 | Intermec Ip Corp. | Automatic data collection device that dynamically wedges data transmitted to data consumers |
| CN1540481A (en) * | 2003-10-31 | 2004-10-27 | 清华大学 | Method for using synergic multiple computer engines based on gridding environment |
| CN1747368A (en) * | 2004-09-10 | 2006-03-15 | 华为技术有限公司 | Restoring method of grid network service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1984006A (en) | 2007-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101252506B (en) | Data transmission system | |
| CN105681462B (en) | It is a kind of based on message routing group system and data communication in shifting method | |
| CN101197851B (en) | Method and system for implementing control of plane centralized type data plane distribution | |
| CN101465824A (en) | Multi-source transmission system and method for instant communication file | |
| EP2883123B1 (en) | Forwarding packet in stacking system | |
| CN111600936A (en) | Asymmetric processing system based on multiple containers and suitable for ubiquitous electric power internet of things edge terminal | |
| Cheng et al. | POTORI: a passive optical top-of-rack interconnect architecture for data centers | |
| CN112769602A (en) | System and method for unified configuration management of white box switch and network operating system | |
| CN100452739C (en) | Optical parallel transmission in usage of grid | |
| CN101098256A (en) | Simple network management protocol based cluster management method and system | |
| CN101340370B (en) | Link selection method and apparatus | |
| Tang et al. | Effective*-flow schedule for optical circuit switching based data center networks: A comprehensive survey | |
| CN116668191B (en) | Internet of things application virtual gateway with data encryption convergence function | |
| WO2008151491A1 (en) | A p2p network system and application method thereof | |
| CN101242412A (en) | A transmission method, device and system for multi-dimension data instance | |
| CN101621528B (en) | Conversation system based on Ethernet switch cluster management and method for realizing conversation passage | |
| CA2439726A1 (en) | Connection-oriented communication network and method of operating | |
| CN100452723C (en) | System and method for managing photon grid network based on mobile agency | |
| CN107480302A (en) | A kind of loose coupling data integration synchronization realizing method based on enterprise-level application scene | |
| CN100562178C (en) | Be applicable to the connection control system and the method for ASON multiple business demand | |
| CN100438472C (en) | Photon grid middleware and its control based on optical network resource allocation on demand | |
| CN113810313B (en) | Processing method and processing device for distributed session message | |
| CN102045294A (en) | Data transmission method and system | |
| CN114401179B (en) | Network element management method, device, equipment and storage medium | |
| CN101163107B (en) | Method of discovering execution route of combined service in wireless self-organizing network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090114 Termination date: 20140530 |