US20080094420A1 - System and method for automatic layout of images in digital albums - Google Patents
System and method for automatic layout of images in digital albums Download PDFInfo
- Publication number
- US20080094420A1 US20080094420A1 US11/957,675 US95767507A US2008094420A1 US 20080094420 A1 US20080094420 A1 US 20080094420A1 US 95767507 A US95767507 A US 95767507A US 2008094420 A1 US2008094420 A1 US 2008094420A1
- Authority
- US
- United States
- Prior art keywords
- page
- album
- image
- images
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
- G06F18/2111—Selection of the most significant subset of features by using evolutionary computational techniques, e.g. genetic algorithms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/771—Feature selection, e.g. selecting representative features from a multi-dimensional feature space
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
- H04N1/00148—Storage
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
- H04N1/00169—Digital image input
- H04N1/00172—Digital image input directly from a still digital camera or from a storage medium mounted in a still digital camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
- H04N1/00169—Digital image input
- H04N1/00175—Digital image input from a still image storage medium
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
- H04N1/00185—Image output
- H04N1/00196—Creation of a photo-montage, e.g. photoalbum
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
- H04N2201/3226—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
- H04N2201/3232—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document of a page, copy or picture number
Definitions
- the invention relates generally to software. More particularly, the present invention relates to software for creating digital photo albums.
- a system for assigning images to album pages includes a mechanism for specifying an initial set of image page assignments to a genetic population and a genetic engine operable to evolve the genetic population to produce a present set of image page assignments.
- the system also includes a page evaluation module operable to test the present set of image page assignments according to an album fitness function to determine an album score and a mechanism for outputting the present set of image page assignments if the album score meets an album threshold value.
- a system for arranging images on an album page includes a mechanism for specifying an initial set of image placement parameters to a genetic population and a genetic engine operable to evolve the genetic population to produce a present set of image placement parameters.
- the system also includes a layout evaluation module, operable to test the present set of image placement parameters with a page fitness function to determine a page score, and a mechanism for outputting the image placement parameters if the page score meets a page threshold value.
- FIG. 1 is an overview diagram of an Albuming Automation System in an illustrative embodiment of the present invention
- FIG. 2 is a flow chart of a genetic algorithm process in an illustrative embodiment of the present invention
- FIG. 3 is a tree diagram of a genetic crossover in an illustrative embodiment of the present invention.
- FIG. 4 is a tree diagram of a genetic mutation in an illustrative embodiment of the present invention.
- FIG. 5 is an array diagram of a genetic crossover in an illustrative embodiment of the present invention.
- FIG. 6 is an array diagram of a genetic mutation in an illustrative embodiment of the present invention.
- FIG. 7 is a system architecture diagram of the page layout module according to an illustrative embodiment of the present invention.
- FIG. 8 is a genome diagram of the page creation module according to an illustrative embodiment of the present invention.
- FIG. 10 is a diagram of an exemplary set of images for manipulation by the page creation module in an illustrative embodiment of the present invention.
- FIG. 11 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 12 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 13 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 14 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 15 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 16 is a page layout example according to an illustrative embodiment of the present invention.
- FIG. 18 is a diagram of the encoding of image rotation information in an illustrative embodiment of the present invention.
- FIG. 20 is an image used in the image placement examples according to an illustrative embodiment of the present invention.
- FIG. 22 is an image used in the image placement examples according to an illustrative embodiment of the present invention.
- FIG. 26 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- FIG. 28 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- FIG. 29 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- FIG. 30 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- FIG. 31 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- FIG. 32 is a diagram of an image placement example according to an illustrative embodiment of the present invention.
- Page layout data is coupled to an output format module 54 that further organizes the output from the automatic page layout module 52 .
- the output of the AAS 2 is produced at module 56 in the form of single page images 58 , entire albums of images 60 , Picture CD media, or other photo delivery media as are understood by those of ordinary skill in the art.
- Coding Genetic algorithms maintain populations of problem solutions. During implementation, these solutions are represented by some sort of data structure.
- the data structure used by a genetic algorithm is known as a genome.
- a data structure is chosen to represent the genome for the problem space and a mapping from the data structure fields to the problem domain is established.
- Common genome data structures used in genetic algorithms includes lists, arrays, and trees.
- the layout problem has a multidimensional problem space.
- FIG. 7 illustrates the overall architecture of a page layout system 124 according to an illustrative embodiment of the present invention.
- the system 124 takes as input a collection of images, which are clustered by events 116 , 118 , 120 , 122 that are to be placed in an album. Being designed to work within the framework of an AAS, the system 124 receives the images that are sorted by event, duplicates and duds have been removed from the collection, and that each image has been assigned an emphasis value indicating the visual appeal of the image.
- the system produces a series of album pages 140 , 142 , 144 , 146 . There are two main system modules corresponding to the tasks outlined above.
- the Page Creator Module 126 is responsible for assigning each image to an album page.
- the Page Creator Module 126 comprises a genetic engine 128 and a page evaluation module 130 , both of which are implemented in software on a processor as is understood by those of ordinary skill in the art.
- the Image Placement Module 132 comprises its own genetic engine 136 and a layout evaluation module 134 , also implemented in software.
- the present invention is implemented using Visual C++ 6.0 running under Windows 95/98/NT.
- the most important implementation task involves the definition of the fitness function.
- the fitness function In defining the fitness for page layout, an approach whereby the fitness is determined directly by the user's visual evaluation would be desirable since the user's subjective interpretation would naturally fit the user's expectations of artistic qualities.
- the AAS is not principally designed for this type of interaction, as one of the goals of the AAS is to minimize the user input once the images to be placed in the album have been input to the system. This does not, however, limit the application of the present invention page layout system in a more interactive environment.
- Page layout is one of the major tasks of those skilled in the art of graphic design.
- the present invention may rely on the principles of graphic design. Upon surveying a number of graphic design texts, including: Amy Arntson, Graphic Design Basics, 3 rd Ed., Harcourt Brace College Publishers, Fort Worth, 1998; Bryan L.
- Balance An equal distribution of weight on the page. This principle refers to the symmetry (or asymmetry) of the page with respect to color, size, shape, and texture.
- Spacing Describes the basic layout of images on a page. Spacing parameters can be used to describe the feel of a layout in terms of white space, or randomness.
- Each of the Page Creator Module 126 and Image Placement Module 132 judge the fitness of solutions based on a number of different criteria from the categories listed above.
- the user's preference for each criterion are input to each of the modules, as was noted respecting the AAS system diagram in FIG. 1 , where user preferences 12 are input to a user database 20 and subsequently decoded and input to these modules. Generally, these are called value preference parameters. In the scope of the AAS, these preference parameter values can be obtained directly by asking the user when the pictures input for album processing. Once established, these preferences are stored as part of a user profile and extracted whenever a particular user requests an album be made.
- the Page Creator Module 126 and Image Placement Module 132 also utilize a set of importance parameters. These importance parameters define how important it is that the system, for a given criterion, obtains a solution where the layout produced evaluates to the exact value of the preference parameter for that criterion. Another way of looking at this is that the importance parameters indicate how much variation the system is allowed with respect to a given criterion. For example, one of the evaluation criteria for the Image Placement Module is white space. A preference of 1.0 indicates that a layout with a larger amount of white space is desired, whereas a preference of 0.0 is indicates that a layout with very little white space is preferred.
- Each module has a number of system parameters. These parameters are not directly used during the layout evaluation process. Instead, these parameters provide flexibility for the modules ensuring that they can be used by different albuming systems and in a variety of albuming situations. Examples of system parameters include page dimensions, resolution of the output device, minimum and maximum number of pages per album, etc.
- the genome for the page creator module makes use of a tree structure as illustrated in FIG. 8 .
- the root of the tree, node 148 represents the entire photo album.
- Interior nodes, 150 , 152 , 154 , 156 , 158 , 160 , 162 , and 164 represent a structure of hierarchical visual groupings of images, which, in turn, are represented by the leaf nodes 166 .
- the first layer of the tree 148 indicates the partitioning of the album into pages. Nodes below the first layer represent visual groupings within pages 150 , 152 , 154 , 156 .
- the notion of this hierarchical visual grouping is illustrated in FIG.
- FIG. 9 which shows the encoding of an album with two pages and a possible layout solution that maintains the visual grouping relationships.
- the album is represented by node 168 . It comprises two pages 170 and a 72 .
- Page One has three images 180 .
- Page Two 172 has three subgroups of images 174 , 176 , and 178 . Each of these subgroups has two images. Subgroup 174 having images 182 , subgroup 176 having images 184 , and subgroup 187 having images 186 .
- Standard crossover and mutation operators for tree structures are used by the Page Creator Module. These operators are illustrated in FIG. 3 and FIG. 4 , as discussed above. Note that the crossover operator may result in a solution that contains images appearing in multiple places within the tree or in solutions where particular images are omitted. These types of offspring are unacceptable and will be assigned a score of 0.0 during evaluation. Because of this, it is to be understood that the probability of performing a crossover has been lowered thus favoring mutation over crossover when constructing the individuals of future generations.
- the determination of fitness used by the Page Creator Module is a combination of a number of factors.
- a solution is evaluated and scored using a number of different criteria.
- the score achieved by the solution is compared to the preference of the user as defined by the preference parameters supplied to the module. This comparison results in a score indicating the suitability of the solution given the preferences of the user.
- the final fitness is obtained by scaling these suitability values on a criterion by criteria basis using the importance parameters, also supplied by the user.
- a score ranging from 0.0 to 1.0 is given to each solution based on the evaluation. Details of the evaluation method for each criterion are given below.
- solutions containing a total number of pages outside the range determined by the system parameters PAGE min -PAGE max are deemed as unfit solutions and given a final fitness of 0.0.
- solutions that place a single image in multiple places, or solutions that do not consider all images are immediately assigned a fitness of 0.0.
- the balance score is a measure of whether there are an equal number of images on each page. A solution with an equal number of images on all of its pages will obtain a perfect score of 1.0. The score is determined by, first, calculating the mean number of images per page. The average deviation from this mean amongst all the pages in the album is determined and scaled by a penalty factor (p balance ). Since a perfect score will have a deviation of 0.0, the final score is obtained by subtracting this scaled deviation value from 1.0.
- each page is given a score based on the percentage of images that fall within chronology range for the page.
- the final score is obtained by computing the average of the page scores over all of the pages in the album.
- Page Unity Score Each page is evaluated to determine the percentage of images that belong to the same event that appear on the page. If all the images on a page belong to the same event or sub-event, the page unity score is 1.0. If several events share the same page, a penalty based on the fraction of events of which images appear on the page is applied. A total page unity score is obtained by adding the positive scores and subtracting penalty scores over all of the pages. This total page score is then calculated by dividing the sum by the number of pages in the album.
- the above scores can be calculated on an event and/or sub-event basis.
- the system allows one to define whether sub-event checking should be performed.
- the final unity score is a linear combination of scores for page and event unity for each evaluation performed. Individual scores are scaled equally in determination of the final unity score.
- a solution represented by a genome
- a score ranging from 0.0 to 1.0.
- a mockup of the page layout is generated by decoding the genome values and positioning rectangles on a two-dimensional grid using a polygon-clipping library.
- the generic polygon clipping library according to the reference: Alan Murta, “ A Generic Polygon Clipping Library ”, Department of Computer Science, University of Manchester, 1999, the contents of which are hereby incorporated by reference thereto, is used.
- This mock layout is used to calculate the various area calculations used in the evaluations.
- the border symmetry score evaluates how closely the edges of the image on the page form a natural boundary. For each edge of the album page (top, bottom, left, right), a border distance is determined by considering the image endpoint closest to the page edge and calculating the distance from the edge to the endpoint. The mean distance amongst the four edges is determined and the average distance from the mean amongst the four edges is calculated. This average is scaled by BDR max and subtracted from 1.0 to generate the final score. Note that if the average distance is greater than BDR max , the solution is flagged as unfit and is assigned a final fitness value of 0.0.
- FIGS. 23 and 24 illustrate the effect of the white space preference. Note that SP max has been set to 75% and SP min has been set to 5%. For FIG. 23 , image 426 , the preference is set to 0.0, and, for FIG. 24 , image 428 , the preference is set to 1.0.
- FIGS. 25 and 26 illustrate the effect of overlap.
- the white space preference is kept constant between the two layouts.
- the value of OL min is set to be 0% whereas the value of OL max is set at 25%.
- image 430 the preference is set to 0.0
- image 432 the preference is set to 1.0.
- FIGS. 27 and 28 The effect of rotation is illustrated in FIGS. 27 and 28 .
- the white space preference is kept constant between the two layouts.
- the overlap preference is set to 0.
- the value of ⁇ max is set at 30 degrees.
- image 434 the preference is set to 0.0
- image 4436 the preference is set to 1.0.
- FIGS. 29 and 30 the effects of rotational balance are shown.
- the rotation preference for both layouts is set to 1.0.
- image 438 the preference is set to 0.0
- image 440 the preference is set to 1.0.
- FIGS. 33 and 34 show the effects of border symmetry.
- the rotation and overlap preferences in this example are set to 0.0.
- image 446 the preference is set to 0.0
- image 448 the preference is set to 1.0.
- FIGS. 35 and 36 the effects of emphasis are illustrated in FIGS. 35 and 36 .
- setting the emphasis preference to 1.0 indicates that the user wants a strong relationship between the size of the image and the emphasis value.
- setting this preference set to 0.0 indicates to the system that this relationship should be reversed. This is clearly the effect shown in these Figs. considering the emphasis values listed for the reference images from FIGS. 19 through 21 .
- image 450 the preference is set to 0.0
- image 4452 the preference is set to 1.0.
- FIGS. 37 and 38 The variety of page layouts that can be produced by the Image Placement Module is best illustrated when modifying several preference parameters at once.
- FIGS. 37 and 38 the preferences for white space, rotation, and overlap are all modified between the two layouts.
- the results are the two layouts illustrated with two entirely different feels to them, FIG. 37 , image 454 , being very traditional and rigid, FIG. 38 , image 456 , having more of a whimsical quality to it.
Abstract
Description
- This application is a continuation of patent application Ser. No. 09/750,858, filed Dec. 29, 2000, entitled, “SYSTEM AND METHOD FOR AUTOMATIC LAYOUT OF IMAGES IN DIGITAL ALBUMS”, the entire disclosure of which is hereby incorporated herein by reference.
- The invention relates generally to software. More particularly, the present invention relates to software for creating digital photo albums.
- Modern photography is moving into the digital age. Even though a large part of the market for photography still utilizes conventional film and processing, the ability to obtain digital files from conventional film is rapidly adapting the conventional film market into the digital image arena. In addition, with the broad acceptance of digital cameras, as well as broad utilization of digital images in Internet applications, the volume of commercial and consumer produced digital image files has increased dramatically in recent years. Suppliers now routinely deliver digital image files to users. Such files may be delivered on storage media, like photo CD's and so forth, or may be delivered via the Internet or through e-mail. The provision of digital images in an organized format, including the preparation of digital image photo albums, is now available in the marketplace, as consumers desire to receive and present their images in a most favorable light. However, there has yet to be introduced an automated system that presents digital images in sophisticated creatively composed fashions.
- There presently exist several software applications, which assist the user in manual creation of digital photo albums. In general, these applications provide the user with a straightforward means of accomplishing the basic task of image organization and page layout, so called ‘albuming’. The drawback with such applications is that they require a large amount of user interaction, which does not yield much improvement over the conventional, paper based albuming techniques of the past. Therefore, the task is less likely to be completed due to the significant amount of effort required to accomplish it.
- Automated albuming systems that offer limited automated layout functions are known in the art. Eastman Kodak Company has developed digital graphic album applications, such as the Kodak Picture Page software, which allow a user to organize digital graphics images into album pages. Typically, users organize images by dates and times, places, subject and so forth. Such software allows the user to annotate the images by entering text, or other data, describing the image. One common approach to automated layout is the use of templates. In this approach, predefined layouts with empty areas are filled in with user images. Thus, the size, placement, rotation, and framing of the images on the page are predetermined. A user or system merely needs to specify which images should be placed in which empty area to complete an album page. This approach is also widely used by many graphic layout design tools (such as Quark) and by Kodak® PicturePage® Software. Templates provide a straightforward and working solution to the layout problem. However, this solution is somewhat limiting as the choice of layouts is bounded by the collection of available templates. Variation in page layout design can only be achieved by the addition of new templates.
- The real challenge in automatic generation of page layouts is allowing a level of creativity in the layouts produced. With the introduction of scrapbooking as a social activity, there has been a recent resurgence of interest in capturing memories and telling stories using scrapbooks. The albums produced by people in these communities can range from extremely basic, where images are laid out using a fixed grid or template, to extremely complex, where images are seemingly scattered randomly on a page. In such layouts, it is very difficult to define, in an objective fashion, what the layout approach is.
- Consequently, a need exists in the art for an automated system and/or method of organizing and generating album and page layouts of digital images.
- The need in the art is addressed by the systems and methods of the present invention. The Automatic Albuming System taught by the present invention is unique, in part, because it produces album pages automatically with minimal input from the user. In an illustrative embodiment, a digital image album layout system is taught. The system comprises a page creator module that has a first genetic engine operable to execute genetic evolution calculations on a first genetic population of image criteria. The page creator module also has a page evaluation module that is operable to test the first genetic population for fitness to album preference criteria. The system also has an image placement module with a second genetic engine that is operable to execute genetic evolution calculations on a second genetic population of page layout criteria. The image placement module also has a layout evaluation module that is operable to test the second genetic population for fitness to page preference criteria.
- The present invention teaches an automated album layout method. The method involves the use of a set of inputs including digital images, graphics, and other 2-dimensional objects. The inventive method comprises the steps of evaluating a grouping of the image objects for distribution into a number of album pages according to a fitness function's parameters of a genetic engine and, assigning each image object to a page based on user preferences, including at least one of: balance, emphasis, chronology, and unity. Then, the page is displayed for user viewing and distribution refining the based on further user action.
- The present invention also teaches an automated layout and presentation method responsive to a set of inputs containing digital images, graphics, and other two-dimensional objects. This method comprises the steps of evaluating the ‘x’ and ‘y’ position coordinates, scale and rotation of each of the input images objects within a page according to fitness function parameters in a genetic engine. Then, a page layout is created based on user preferences including at least one of: white space, overlap, rotation, spatial balance, rotational balance, border symmetry, and emphasis. Then, the page layout is displayed for user viewing, and refining the page layout based on further user action. Finally, the page layout is formatted for printing.
- In an illustrative embodiment, a system for assigning images to album pages is taught. The system includes a mechanism for specifying an initial set of image page assignments to a genetic population and a genetic engine operable to evolve the genetic population to produce a present set of image page assignments. The system also includes a page evaluation module operable to test the present set of image page assignments according to an album fitness function to determine an album score and a mechanism for outputting the present set of image page assignments if the album score meets an album threshold value.
- In another illustrative embodiment, a system for arranging images on an album page is taught. This system includes a mechanism for specifying an initial set of image placement parameters to a genetic population and a genetic engine operable to evolve the genetic population to produce a present set of image placement parameters. The system also includes a layout evaluation module, operable to test the present set of image placement parameters with a page fitness function to determine a page score, and a mechanism for outputting the image placement parameters if the page score meets a page threshold value.
-
FIG. 1 is an overview diagram of an Albuming Automation System in an illustrative embodiment of the present invention; -
FIG. 2 is a flow chart of a genetic algorithm process in an illustrative embodiment of the present invention; -
FIG. 3 is a tree diagram of a genetic crossover in an illustrative embodiment of the present invention; -
FIG. 4 is a tree diagram of a genetic mutation in an illustrative embodiment of the present invention; -
FIG. 5 is an array diagram of a genetic crossover in an illustrative embodiment of the present invention; -
FIG. 6 is an array diagram of a genetic mutation in an illustrative embodiment of the present invention; -
FIG. 7 is a system architecture diagram of the page layout module according to an illustrative embodiment of the present invention; -
FIG. 8 is a genome diagram of the page creation module according to an illustrative embodiment of the present invention; -
FIG. 9 is a diagram illustrating the encoding of the genome by the page creation module according to an illustrative embodiment of the present invention; -
FIG. 10 is a diagram of an exemplary set of images for manipulation by the page creation module in an illustrative embodiment of the present invention; -
FIG. 11 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 12 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 13 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 14 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 15 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 16 is a page layout example according to an illustrative embodiment of the present invention; -
FIG. 17 is a diagram of the genome encoding for the image placement module in an illustrative embodiment of the present invention; -
FIG. 18 is a diagram of the encoding of image rotation information in an illustrative embodiment of the present invention; -
FIG. 19 is an image used in the image placement examples according to an illustrative embodiment of the present invention; -
FIG. 20 is an image used in the image placement examples according to an illustrative embodiment of the present invention; -
FIG. 21 is an image used in the image placement examples according to an illustrative embodiment of the present invention; -
FIG. 22 is an image used in the image placement examples according to an illustrative embodiment of the present invention; -
FIG. 23 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 24 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 25 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 26 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 27 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 28 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 29 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 30 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 31 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 32 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 33 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 34 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 35 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 36 is a diagram of an image placement example according to an illustrative embodiment of the present invention; -
FIG. 37 is a diagram of an image placement example according to an illustrative embodiment of the present invention; and -
FIG. 38 is a diagram of an image placement example according to an illustrative embodiment of the present invention. - While the present invention is described herein with reference to illustrative embodiments for particular applications, it should be understood that the invention is not limited thereto. Those having ordinary skill in the art and access to the teachings provided herein will recognize additional modifications, applications, and embodiments within the scope thereof and additional fields in which the present invention would be of significant utility.
- The principle objective of albuming automation is to automate the album production process using various image science algorithms and techniques. The final step of this automated process is to layout images within an album in a manner pleasing to the user.
- The present invention teaches an innovative and flexible system for automatic album page layout that makes advantageous use of genetic algorithms. The system is divided into two modules, a Page Creator Module which is responsible for distributing images amongst various album pages and an Image Placement Module which positions images on individual pages. Each module incorporates basic graphic design principles such as balance, emphasis, rhythm, and unity, in determining fitness for page layout solutions.
- A complete albuming automation system utilizes various image science algorithms and techniques including advanced event clustering, dud detection, image appeal and automatic page layout. In an illustrative embodiment, the emphasis of such a system is for a “DAFY” (Do-it-All-For-You) like product, where the user inputs a collection of images and the system produces an album (a collection of images) with minimal input from the user. It will be understood by those of ordinary skill in the art, that the term ‘images’ encompasses a much broader scope than the conventional photograph, even though the album concept stems from the traditional photographic album. In the modern digital world, images include computer generated graphics, bitmaps, photographs, computer altered photographs, video still frames, scanned images, various forms or artwork, text, background materials, and even video clips, animation, and computer generated time variant materials.
- An overall functional diagram of an illustrative embodiment Albuming Automation System (‘AAS’) 2 is depicted in
FIG. 1 . One important function of theAAS 2 is to layout images within an album page. This function is the last major operation before an album page can be rendered for display, printing, or transfer to any another medium. In the illustrative embodiment ofFIG. 1 , theAAS 2 receivesinput 4 from one of a variety of image sources, which includesconventional film 6,flash memory cards 8,video media 10, and other images sources as are understood by those of ordinary skill in the art. Another basic input to theAAS 2 is the user specifiedpreferences 12, which will be more fully discussed hereinafter.Film 6 input is first digitized 14 for subsequent processing. Flashmemory card media 8 is decoded 16.Video media 10 is also digitized 18. Once a common digital format is established, the image information is processed to recover anymetadata information 22 that may be available. Metadata can be found in Internet originated images, and may be supplied by the user or other individual. For example, APS cameras allow for inclusion of metadata related to the date and time the image was taken, the image format, and other technical parameters related to the exposure of the image.User preferences 12 are stored in auser preference database 20 and are subsequently decoded 24 for use in processing the user preferences along with the image information. By maintaining a database of user preferences, subsequent use of the system by a particular user can be made more efficient and convenient. The system is able to recall the user preferences and apply them to new batches of images, and can offer the user an opportunity to amend previous preferences as well. - The core processing of the
AAS 2 includesseveral functions 26 that discriminate images and information for subsequent page layout. These include clustering of images byevent 28, detection ofdud images 30, detection ofduplicate images 32, recognition of facial features and certainother objects 34, audio to textconversion 36, andvideo summarization 38. The reduced and refined image information is then coupled to a second group of core process functions 40 that further refine the image content information. These functions include selection of the best image perpage 42, automatic image cropping 44, association of particular images withancillary content 46, association withevent description information 48, and association with caption andannotation information 50. The refined image information is coupled to the automaticpage layout process 52, which will be more fully described hereinafter. Page layout data is coupled to anoutput format module 54 that further organizes the output from the automaticpage layout module 52. Finally, the output of theAAS 2 is produced atmodule 56 in the form ofsingle page images 58, entire albums ofimages 60, Picture CD media, or other photo delivery media as are understood by those of ordinary skill in the art. - The present invention teaches a flexible system for generation of album page layouts. The system makes use of genetic algorithms, a class of search and optimization algorithms that are based on the concepts of biological evolution. For a more detailed reference to genetic algorithms, see; J. Holland, Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor, 1975, and, D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989, the contents of which are hereby incorporated by reference thereto. The system is comprised of two major modules, the first that distributes images amongst a set of album pages, and the second that positions the images on each individual page. These modules are called the Page Creator Module and the Image Placement Module respectively. Each module takes a genetic approach to its task.
- The overall function of the
Page Layout System 124 is straightforward. Given a set of images to be placed in an album, a page layout algorithm must distribute the images amongst a set of pages and then layout the images on each individual page. Working within the framework of the AAS, the following information is available to thepage layout system 124 on an image by image basis: -
- Event Clustering—Images are grouped by event and sub-event
- Dud/Duplicate Detection—Duplicate and dud detection are performed on the images prior to being submitted to the page layout system. Thus, the list of images supplied as input to the page layout process represent all the images that will be contained within the final album.
- Emphasis/Image Appeal—Images have an associated image appeal or emphasis value. This value is a measure of relative importance and is be used as a guide in determining the emphasis that an image will be given when placing it on an album page.
- Chronology—The chronology of the images to be placed in the album is known.
- The
page layout system 124 performs two separate, yet equally important tasks.Page creation 126, given a set of images, the system distributes these images amongst a set of album pages, such that each image is assigned a page upon which the image will appear. And,image placement 132, once the images have been assigned to pages, each individual page is laid out by positioning the images assigned to it. Therefore, for each image, placement, rotation, and scaling of the image on the page are assigned. - With regard to album layout according to the present invention, an important advantage is that the Automated Albumimg System produces albums that more closely resemble scrapbooks as opposed to a simple collection of pictures. Achieving this goal in an automated fashion is a significant accomplishment because the means by which creative scrapbookers generate page layouts for their albums is usually not easily expressed in an objective fashion. The creation of a scrapbook is primarily a subjective and artistic task. Few, if any, concrete rules exist in the scrapbook generation process, and those that do, tend to be individual based on personal preferences. The subjective nature of creative page layout poses a real challenge to any page layout system. In particular, template based layout approaches are somewhat limiting since the range of possibilities for a page layout are bounded by the collection of available templates.
- The present invention employs a novel approach to page layout by employing genetic algorithms, which are a class of adaptive methods that can be used to solve search and optimization problems involving large search spaces. The search is performed using a simulated evolution (survival of the fittest). These algorithms maintain and manipulate “generations” of potential solutions or “populations”. With each generation, the best solutions (as determined by a problem specific fitness function) are genetically manipulated to form the solution set for the following generation. As in real evolution, solutions can be combined (via mating/crossover) or undergo random mutation. In addition, inferior solutions can, by chance, survive from generation to generation.
- The genetic algorithm search process is performed in an iterative manner as illustrated in
FIG. 2 . The process starts atstep 64, and then an initial population of solutions is generated atstep 66. This initialization is typically done through random placement of the image population, although there is nothing prohibiting in using some sort of heuristic for this process. Once a population is established, individual solutions are judged atstep 68 based on a fitness function and suitable individuals are chosen for mating and mutation. Next, atsteps step 70, and crossover,step 72. A new generation of solutions is then created from these newly generated offspring by inserting them into the population atstep 74. It is noted that there are a number of different variants used when constructing a new generation of solutions. In some cases, individuals chosen for mating and mutation are included in this next generation of solutions. Note also, that unlike biological mating, new solutions can be generated from the crossover of two or more suitable individuals. The process is iterated until a defined stopping criterion is met atstep 76, in which case the process finishes atstep 78. The stopping criteria will be more fully discussed hereinafter. - When applying a genetic algorithm to a given problem, three major tasks must be performed:
- 1) Coding—Genetic algorithms maintain populations of problem solutions. During implementation, these solutions are represented by some sort of data structure. The data structure used by a genetic algorithm is known as a genome. In the coding task, a data structure is chosen to represent the genome for the problem space and a mapping from the data structure fields to the problem domain is established. Common genome data structures used in genetic algorithms includes lists, arrays, and trees.
- 2) Definition of Genetic Operators—New solutions are created via crossover and mutation of individuals from previous generations. Given a particular genome structure, the means for performing these operations must be defined. During crossover, one or more children solutions are derived from two or more parents. With mutation, new individuals are generated by mutation of a single solution. There are standard crossover and mutation operators available for genomes encoded using commonly used data structures like lists, arrays, and trees.
- 3) Fitness—The most challenging and application specific task in applying genetic algorithms to a problem domain is in the definition of a fitness function. The fitness function is responsible for judging individual solutions and returning a score based on its evaluation. In essence, the fitness function defines the difference between a good solution to a problem and a bad one. Much care must be taken in defining the fitness function, as the genetic algorithm will converge on solutions deemed “fit” by this function.
-
FIG. 3 illustrates a genetic crossover operation fortrees random nodes trees FIG. 4 illustrates a genetic mutation operation fortree 92. Nodes from thetree Tree 94 illustrates the change fromtree 92 when this swap occurs. -
FIG. 5 andFIG. 6 illustrate similar genetic crossover and mutation operators, but in the case of an array type data structure. InFIG. 5 , for a crossover, aposition 101 within thearray 100 is randomly selected and the array values 104 and 106 after the chosenposition 101 are swapped between the two parents, resulting incrossover array 102. Similarly, inFIG. 6 , tworandom array elements single parent 108 and their respective positions are swapped to produceresultant array 110. - An impetus to using genetic algorithms for page layout came from an application for creating artistic textures. This source was Karl Sims who uses genetic algorithms for creating 2D textures as articulated in the reference; Karl Sims, “Artificial Evolution for Computer Graphics”, Proceedings of SIGGRAPH 91, pp 319-328, the contents of which are hereby incorporated by reference thereto. The motivations behind his work was mostly artistic whereby the artist directly determined the fitness of each solution by visual inspection. The system allowed for random exploration of the texture space with solutions converging based on the likes and dislikes of the artist.
- Page layout has also been found to be more of an artistic task rather than a mechanical one. Genetic algorithms are appropriate for such artistic tasks since, unlike other more brute force algorithms, the genetic algorithm does not attempt to mimic or model any particular process by which solutions are created. Instead, solutions are generated randomly and are evaluated after the fact. This is analogous to the way creative scrapbookers approach the layout problem. While scrapbookers can't usually explain the process by which they generated their page layouts, they certainly know a good layout when they see one.
- The layout problem has a multidimensional problem space. Considering the image distribution task discussed herein before, there are at least four parameters that must determined for each image that is to be positioned on a page. These are the ‘x’ and ‘y’ positions, the rotation angle of each image, and the size scaling of each image. Multiplying that by the number of images to be placed results in a solution space ranging from four dimensions, in the case of a single image to be placed, to as high as forty dimensions, in the case of ten images to be placed. Genetic algorithms have been proven successful for problems with similarly large dimensional solution spaces, hence they are suitable for automatic albuming system applications.
-
FIG. 7 illustrates the overall architecture of apage layout system 124 according to an illustrative embodiment of the present invention. Thesystem 124 takes as input a collection of images, which are clustered byevents system 124 receives the images that are sorted by event, duplicates and duds have been removed from the collection, and that each image has been assigned an emphasis value indicating the visual appeal of the image. The system produces a series ofalbum pages Page Creator Module 126 is responsible for assigning each image to an album page. The page groupings created by this module are passed to theImage Placement Module 132, which positions the images on each individual page. Page layouts are specified in a textual form using an Extensible Markup Language (XML) format. Details about this format are available from: World Wide Web Consortium, Extensible Markup Language (XML) 1.0, http://www.w3.org/xml, February 1998, the contents of which are hereby incorporated by reference thereto. These textual descriptions are interpreted by aXML compiler 138 that creates composite images corresponding to each completed album page. Both thePage Creator Module 126 andImage Placement Module 132 make use genetic algorithms and consists of their own separate genetic engines, 128 and 136 respectively. More particularly, thePage Creator Module 126 comprises agenetic engine 128 and apage evaluation module 130, both of which are implemented in software on a processor as is understood by those of ordinary skill in the art. TheImage Placement Module 132 comprises its owngenetic engine 136 and alayout evaluation module 134, also implemented in software. In an illustrative embodiment, the present invention is implemented using Visual C++ 6.0 running under Windows 95/98/NT. Although porting to other platforms is readily feasible, as well as implementation in other programming languages and operation on other platforms, understood by those of ordinary skill in the art. - The most important implementation task involves the definition of the fitness function. In defining the fitness for page layout, an approach whereby the fitness is determined directly by the user's visual evaluation would be desirable since the user's subjective interpretation would naturally fit the user's expectations of artistic qualities. The AAS, however, is not principally designed for this type of interaction, as one of the goals of the AAS is to minimize the user input once the images to be placed in the album have been input to the system. This does not, however, limit the application of the present invention page layout system in a more interactive environment.
- Due to the subjective nature of album evaluation, some indication of the kind of layouts to be produced is required. Page layout is one of the major tasks of those skilled in the art of graphic design. In defining a means for a user to specify album layout preference, the present invention may rely on the principles of graphic design. Upon surveying a number of graphic design texts, including: Amy Arntson, Graphic Design Basics, 3rd Ed., Harcourt Brace College Publishers, Fort Worth, 1998; Bryan L. Peterson, Using Design Basics To Get Creative Results, Northern Lights Books, Cincinnati, Ohio, 1997, and Lori Siebert and Lisa Ballard, Making a Good Layout, Northern Light Books, Cincinnati, Ohio, 1992, the contents of which are hereby incorporated by reference thereto, the preferred embodiment of the present invention extracted a number commonly mentioned principles used in evaluation of layout design, which include:
- 1) Balance—An equal distribution of weight on the page. This principle refers to the symmetry (or asymmetry) of the page with respect to color, size, shape, and texture.
- 2) Spacing—Describes the basic layout of images on a page. Spacing parameters can be used to describe the feel of a layout in terms of white space, or randomness.
- 3) Chronology—Describes whether the placement of images on the page matches with the temporal order in which the pictures were taken.
- 4) Emphasis—What stands out most gets noticed first. Emphasized elements will be the focal point of a page. Although generally achieved using element size, emphasis can also be maintained by use of color, shape, framing, and texture.
- 5) Unity—Elements that belong together look like they belong together. Unity is achieved by grouping, repetition, and the use of grids (a subdivision of space into rows, columns, and margins).
- Each of the
Page Creator Module 126 andImage Placement Module 132 judge the fitness of solutions based on a number of different criteria from the categories listed above. The user's preference for each criterion are input to each of the modules, as was noted respecting the AAS system diagram inFIG. 1 , whereuser preferences 12 are input to auser database 20 and subsequently decoded and input to these modules. Generally, these are called value preference parameters. In the scope of the AAS, these preference parameter values can be obtained directly by asking the user when the pictures input for album processing. Once established, these preferences are stored as part of a user profile and extracted whenever a particular user requests an album be made. - In addition to preference parameters, the
Page Creator Module 126 andImage Placement Module 132 also utilize a set of importance parameters. These importance parameters define how important it is that the system, for a given criterion, obtains a solution where the layout produced evaluates to the exact value of the preference parameter for that criterion. Another way of looking at this is that the importance parameters indicate how much variation the system is allowed with respect to a given criterion. For example, one of the evaluation criteria for the Image Placement Module is white space. A preference of 1.0 indicates that a layout with a larger amount of white space is desired, whereas a preference of 0.0 is indicates that a layout with very little white space is preferred. Setting the preference for white space to be 0.0 with an importance of 1.0, the system will do everything it can to assure that the layout has as little white space as possible. Note that this is not the same as saying that the importance of white space is 0.0. In the above example, white space is very important, it's just that the user wants very little of it. An importance of 0.0 indicates to the system that it should not even pay attention to the white space preference and, as a result produce a solution with as much or little white space as the system deems appropriate. - Each module has a number of system parameters. These parameters are not directly used during the layout evaluation process. Instead, these parameters provide flexibility for the modules ensuring that they can be used by different albuming systems and in a variety of albuming situations. Examples of system parameters include page dimensions, resolution of the output device, minimum and maximum number of pages per album, etc.
- The function of the Page Creator Module is to place each image onto an album page. Alternately, this module can be thought of as being responsible for assigning to each image a page number, where this number corresponds to the page on which the image will be placed. The Page Creator Module makes use of a number of system parameters. These parameters are listed in Table 1 below. As noted above, the values for these system parameters are set before the task of page creation commences. The symbol assigned to each parameter will be used in subsequent sections for referencing the values of these parameters.
TABLE 1 System parameters for Page Creator Module Parameter Symbol Description Emphasis range σemphasis Determines the range of “acceptable” emphasis values for the sum of the emphasis values placed on a given page Minimum Pages PAGEmin The minimum number of pages allowable in the album Maximum Pages PAGEmax The maximum number of pages allowable in the album Penalty factor pbalance A scale factor for penalizing unbalanced for balance solutions - The genome for the page creator module makes use of a tree structure as illustrated in
FIG. 8 . The root of the tree,node 148, represents the entire photo album. Interior nodes, 150, 152, 154, 156, 158, 160, 162, and 164 represent a structure of hierarchical visual groupings of images, which, in turn, are represented by theleaf nodes 166. However, it is equally suitable to replace this tree based encoding with a simpler data structure based on arrays. The first layer of thetree 148 indicates the partitioning of the album into pages. Nodes below the first layer represent visual groupings withinpages FIG. 9 , which shows the encoding of an album with two pages and a possible layout solution that maintains the visual grouping relationships. The album is represented bynode 168. It comprises twopages 170 and a72. Page One has threeimages 180. Page Two 172 has three subgroups ofimages Subgroup 174 havingimages 182,subgroup 176 havingimages 184, and subgroup 187 havingimages 186. - Standard crossover and mutation operators for tree structures are used by the Page Creator Module. These operators are illustrated in
FIG. 3 andFIG. 4 , as discussed above. Note that the crossover operator may result in a solution that contains images appearing in multiple places within the tree or in solutions where particular images are omitted. These types of offspring are unacceptable and will be assigned a score of 0.0 during evaluation. Because of this, it is to be understood that the probability of performing a crossover has been lowered thus favoring mutation over crossover when constructing the individuals of future generations. - The determination of fitness used by the Page Creator Module is a combination of a number of factors. First, a solution is evaluated and scored using a number of different criteria. For each criterion, the score achieved by the solution is compared to the preference of the user as defined by the preference parameters supplied to the module. This comparison results in a score indicating the suitability of the solution given the preferences of the user. Finally, the final fitness is obtained by scaling these suitability values on a criterion by criteria basis using the importance parameters, also supplied by the user.
- In the following paragraphs, the fitness evaluation process is discussed in more detail. First a discussion of the evaluation criteria and judging procedure is given. Then the means by which the final fitness is obtained is discussed.
- Evaluation Criteria—Solutions for the Page Creator Module are evaluated on the following four criteria:
-
- Balance—an evaluation of the balance of the image distribution with respect to the number of images on each page.
- Emphasis—an evaluation of whether image emphasis values are equally distributed amongst album pages.
- Chronology—an evaluation of how well the distribution of images on sequential pages matches the chronology of the images.
- Unity—an evaluation of whether images belonging to the same event and/or sub-event are grouped on the same or subsequent pages.
- For each criterion, a score ranging from 0.0 to 1.0 is given to each solution based on the evaluation. Details of the evaluation method for each criterion are given below.
- Note that solutions containing a total number of pages outside the range determined by the system parameters PAGEmin-PAGEmax, are deemed as unfit solutions and given a final fitness of 0.0. Similarly, solutions that place a single image in multiple places, or solutions that do not consider all images are immediately assigned a fitness of 0.0.
- Evaluation of Balance—The balance score is a measure of whether there are an equal number of images on each page. A solution with an equal number of images on all of its pages will obtain a perfect score of 1.0. The score is determined by, first, calculating the mean number of images per page. The average deviation from this mean amongst all the pages in the album is determined and scaled by a penalty factor (pbalance). Since a perfect score will have a deviation of 0.0, the final score is obtained by subtracting this scaled deviation value from 1.0.
- Evaluation of Emphasis—In considering emphasis, the evaluation method attempts to judge how equally distributed the emphasis is amongst the pages in the album. It is undesirable to have too much emphasis on a single page since this will limit the capability of the image placement module to adequately emphasize the images that have high emphasis values. At the same time, a page with too little emphasis on a page will force the image placement module to emphasize images that don't properly deserve emphasis.
- In the evaluation of emphasis, it is assumed that the sum of the emphasis values for all of the images on a “perfect” page will equal 1.0. This ideal value of 1.0 could be replaced by some value based on the average of the emphasis values amongst all of the images. An evaluation score for emphasis is obtained by calculating the percentage of pages whose emphasis sum amongst all of the images placed on it, are within an “acceptable” range. This “acceptable range” is defined to be 1.0±σemphasis.
- Evaluation of Chronology—In evaluating chronology, each solution is judged based on how closely the order of the images as presented on the pages match that of the chronology of the images. To perform this evaluation, first define the notion of a “chronology range”. This range is defined as the range of chronology values for images that should appear on the page if the album was perfectly ordered in time. For example, given an album with three pages, with two images on the first page, four images on the second page, and three images on the final page, the chronology range for each page would be given according to Table 2:
TABLE 2 Chronology ranges Page Chronology range Page 1 1-2 Page 23-6 Page 37-9 - When performing the evaluation for chronology, each page is given a score based on the percentage of images that fall within chronology range for the page. The final score is obtained by computing the average of the page scores over all of the pages in the album.
- Evaluation of Unity—Evaluation of unity is performed by considering the images belonging to events and sub-events and their proximity to each other when constructing the pages for the album. Two types of scores are computed when evaluating unity:
- 1) Page Unity Score—Each page is evaluated to determine the percentage of images that belong to the same event that appear on the page. If all the images on a page belong to the same event or sub-event, the page unity score is 1.0. If several events share the same page, a penalty based on the fraction of events of which images appear on the page is applied. A total page unity score is obtained by adding the positive scores and subtracting penalty scores over all of the pages. This total page score is then calculated by dividing the sum by the number of pages in the album.
- 2) Event Unity—Each event is evaluated to determine the percentage of images in the same event that fall on the same or subsequent pages. For each page that images of a particular event appears, the fraction of images from that page that belong to that event (or sub-event) is determined. If an event totally dominates a page, a score of 1.0 is added to the total Event Unity score. Otherwise, the fraction of images on that page not belonging to the event in question is applied as a penalty and subtracted from the total Event Unity Score. It is acceptable for an event or sub-event to scan multiple pages. For those events that do, a bonus is applied when the pages on which the event images appear are subsequent. A penalty, based on the distance between the pages on which the images appear, is applied if this is not the case. Finally, the final event unity score is scaled by the best possible score given the number of events and the arrangement of the images amongst the pages.
- The above scores can be calculated on an event and/or sub-event basis. The system allows one to define whether sub-event checking should be performed. The final unity score is a linear combination of scores for page and event unity for each evaluation performed. Individual scores are scaled equally in determination of the final unity score.
- Calculation of Final Fitness—The evaluation algorithms listed above provide raw scores for each one of the evaluation criteria. In determining the final fitness of the solution, both the preferences and the importance of each criteria, as specified by the preference and importance parameters supplied by the user, must also be considered. To determine how well a solution meets the preferences of the user, the difference between the user preference and the raw score is calculated for each criterion. These differences are subtracted from 1.0 resulting in a fitness score for each criteria such that a fitness score of 1.0 indicates a good match with user preferences and a score of 0.0 indicated a bad match with user preferences.
- The values of the importance parameters are used to scale the contribution of each criterion to the final fitness score. Given a set of importance parameter values, the best possible score obtainable for a given run of the algorithm can be calculated by assuming the fitness score for each criterion to be perfect (i.e. equal to 1.0). The final fitness score is determined by scaling each of the actual fitness scores for each criteria by the corresponding importance parameter value, summing the results from all of the criteria and dividing this sum by the best possible fitness value obtainable. This final fitness score can be summarized mathematically as:
where ‘I’ represents the set of importance parameters value (1 per criteria), ‘P’ represents the set of preference parameter values (1 per criteria) and ‘E’ represents the set of raw evaluations scores as determined by the procedures outlined above (1 per criteria). - To illustrate the relationship of these functions and clarify the process generally, what follows are several examples of using the Page Creator Module on a group of images. In each of the examples, the image set presented in
FIG. 10 is used. While the images themselves are not present, the data is derived from actual user images and represents a typical user role of film consisting of 40 images taken from a variety of different events. To facilitate viewing the effects of the various evaluation criteria, the images have been labeled with a chronology index (C), giving the relative ordering of the picture with respect to the entire set. An assigned emphasis value (E) is also supplied with each image. Thus, referring toFIG. 10 , groups of images from six events are illustrated. Event One 188 includes twelve imaged, chronologically C:0 through C: 11. Event two 190 includes nine images, chronologically C:12 through C:20. Event three includes eight images, chronologically C:21 through C:28. Event four 194 includes six images, chronologically C:29 through C:34. Event five 196 includes four images, chronologically C:35 through C:38. Finally, event six 198 includes one image, chronologically C:39. Each image has a emphasis value which can be determined by reference toFIG. 10 . - In each of tests illustrated in
FIG. 11 throughFIG. 16 , the effects due to a single criterion are illustrated. In these tests, the preference and importance parameters for the criterion in question are set to 1.0 and the importance parameters for all other criteria is set to 0.0. In each test, the genetic algorithm is run for 10,000 generations, producing solutions with scores ranging from 0.75 through 1.0. -
FIG. 11 illustrates the effect of chronology. A good chronology score is generally not difficult to achieve given that the images are originally presented in chronological order. In fact, the trivial solution is a layout of six pages, where each page contains the images from a single event. InFIG. 11 , PAGEmin is set to ten as to assure that the Page Creator Module performs some non-trivial work (e.g., it cannot find the trivial solution as six pages is prohibited by this parameter). In the figure, the chronology index is displayed on each image. Referring toFIG. 11 , it can be seen that ten pages are output from the Page Creation Module, identified byreference numbers -
FIG. 12 illustrates the effects of unity Since unity forces images of the same event to be placed next to each other, as with the chronology case, the trivial solution of a six page album with one event per page will receive a perfect score of one. For the test illustrated inFIG. 12 , PAGEmin is set to 10, once again, to allow the module to perform some non-trivial work. This particular example, though not perfect, tends to keep events together, if not on the same page, on subsequent pages. The output comprised 16 pages.Page Page reference numerals Page Pages Page Pages reference numerals Page Page Page event 2, two images fromevent 4, and five images from event four. In this page, all common event images are adjacent to one another. Finally,page event 4. This solution received a score of 0.8. - The effects of balance are illustrated in
FIG. 13 . In this example, no constraints are placed on the minimum or maximum number of pages. Note that pages 2, 4, 5, 6, 7, 9, 13, 16, and 17, identified byreference numerals Pages reference numerals Page - The effects of emphasis are illustrated in
FIG. 14 . As with the balance test, no constraints were placed on the minimum or maximum number of pages. Note that emphasis values are printed for each image, along with the chronology value. Seventeen pages were generated, identified byreference numerals - Considering the solutions above, it is clear that unity and chronology tend to lump images together on the same page, whereas emphasis and balance tend to favor solutions where images are more spread out amongst pages. An interesting compromise is reached when combining the effects of two criteria. In
FIG. 15 , equal importance values are assigned to emphasis and unity (with the importance of balance and chronology set to 0.0). Twenty-one pages, identified byreference numerals Page 15 inFIG. 12 ). This same observation is also evident inFIG. 16 where balance and chronology are assigned equal importance (with the importance of unity and emphasis set to 0.0). In this case, some sense of unity is achieved indirectly since events are presented in chronological order. Note inFIG. 16 that thirteen pages are generated, identified byreference numerals - Now considering the Image Placement Module (
item 134 inFIG. 7 ), its role is to position images on individual album pages once the Page Creator Module has assigned images to particular pages, as discussed above. Given a set of ‘N’ images to be placed on an individual album page, this module will determine, for each image, the following positioning parameters: -
- ‘x’ position—The x coordinate of the center of the image.
- ‘y’ position—The y coordinate of the center of the image.
- ‘s’ Scale Factor—The amount of scaling to be applied to the image. Note that the aspect ratios of the images are preserved. Thus, equal scaling is performed in both the horizontal and vertical directions.
- ‘θ’ Rotation—Amount of rotation about the center vertical axis of the image.
- In the following discussions, the positioning parameters (x, y, s, θ) are be used to indicate the calculated position of an image on the page with each member of the four factors corresponding to a respective value listed above. A set of four positioning parameters define, and are referred to as, the image position.
- In performing the solution encoding and solution fitness evaluation, the Image Placement Module makes use of a number of system parameters. The values of these parameters will vary dependent on the application using the module and are set before layouts are processed by the module. A list of these system parameters is given in Table 3. The symbol assigned to each parameter will be used for reference hereinafter.
TABLE 3 System parameters for Image Placement Module Parameter Symbol Description Album Page Width Wpage The width of the album page in pixel units (or physical units). Album Page Height Hpage The height of the album page in pixel units (or physical units). Maximum Rotation θmax Maximum allowable rotation about the center vertical axis of an image. Minimum Scaling smin Minimum allowable scale factor that can be applied to an image. Maximum Scaling smax Maximum allowable scale factor that can be applied to an image. Minimum white WSmin Minimum allowable amount of white space on the page, space expressed as a percentage of the total page area. Maximum white WSmax Maximum allowable amount of white space on the page, space expressed as a percentage of the total page area. Minimum overlap OLmin Minimum allowable amount of overlap, expressed as a percentage of image area. Maximum overlap OLmax Maximum allowable amount of overlap, expressed as a percentage of image area. Border deviation BDRmax Maximum deviation allowed in definition of a natural border created by the image placements. Rotation standard σθ Standard deviation used in Gaussian scaling of rotation. deviation - A floating point array is used as the genome for the image placement module. Reference is directed to
FIG. 17 , which is an illustration of such anarray 388 in an illustrative embodiment of the present invention. Each element of the array corresponds to an individual positioning parameter. The first four elements, 394, 396, 398, and 400 specify the Position (x), Position (y), Scaling (s), and Rotation (θ) for the first image 390. Each image is similarly defined through the Nth image 392. The complete genome is comprised of the image positions of all images to be placed. Thus, for a page with N images, the length of the genome is four times N. - All elements of the array are floating point values ranging from 0.0 to 1.0. This is to assure that all genes (i.e. array elements) are considered equal when performing genetic operations. The value of the genes for a given image are be referred as genex, geney, genes, and geneθ for the value corresponding to the x position, y position, scaling, and rotation respectively. The mappings from these array element values to positioning parameter values for each positioning parameter are given hereinafter.
- The x and y positioning parameters give the placement of the center of a given image on the album page. In the genome, this is expressed relative to the total height and width of the page with the origin being the upper-left corner of the page. Appropriate calculations are made by the Image Placement Module to assure that an image placement calculated from given gene values will not result in any part of the image being placed off the boundaries of the album page. The mapping from the gene values for x and y positioning to actual x and y position on the page can thus be given by:
- where the width and height of the image in pixels are given by Wimage and Himage respectively.
- Scaling is expressed in the gene by a linear ramp between the minimum allowable scaling and the maximum allowable scaling. The mapping from gene value to actual scale factor is thus given by:
s=s min+(genes·(s max −s min)) - The rotation positional parameter gives the rotation of an image about the center of the image with respect to the vertical axis of that image. Reference is directed to
FIG. 18 . Animage 402 may be rotated to acounter-clockwise position 406 or aclockwise position 404. Rotation is encoded in the gene by a Gaussian scaling between maximum allowable rotation in the negative (counter-clockwise) direction through angle 408 and the maximum allowable rotation in the positive (clockwise) direction throughangle 410. Thus, the vertical axis of theimage 412, is rotated through the aforementioned angles to aposition - The mapping from gene value to actual rotation angle can thus be given by:
where SIGN (x) equals 1 if x is positive, −1 if x is negative and 0 otherwise. - In an illustrative embodiment, standard crossover and mutation operators for arrays are used by the Image Placement Module. These operators are illustrated in
FIG. 5 andFIG. 6 . In additional, a value mutation operator is defined and also used alternately by the module. This value mutation operator will choose a random gene within the array and deviate it's value by a random amount. Care is taken to assure that the gene value remains between 0.0 and 1.0 after the mutation. - Similar to the Page Creator Module, the fitness for the Image Placement Module is a combination of raw evaluations in a number of given criteria considered with respect to the preference and importance parameters supplied from the user preference database. The method for evaluation of fitness is outlined in the following discussion. The evaluation criteria for layout solutions include seven different criteria. These criteria can be categorized as follows.
-
- Spatial Criteria
- White space—an evaluation of the amount of white space on the page.
- Overlap—an evaluation of the amount of image overlap on a page.
- Rotation—an evaluation of the amount of image rotation on a page.
- Balance Criteria
- Spatial Balance—an evaluation of how equally distributed the images are on the page.
- Rotational Balance—an evaluation of how equally balanced the rotation of images are.
- Border Symmetry—an evaluation of how well the edges of the images on the page form a natural border.
- Emphasis—an evaluation of how well the scaling of images relate to image emphasis values.
- Spatial Criteria
- For each criterion, a solution, represented by a genome, is given a score ranging from 0.0 to 1.0. In order to perform the evaluations, a mockup of the page layout is generated by decoding the genome values and positioning rectangles on a two-dimensional grid using a polygon-clipping library. In an illustrative embodiment of the present invention, the generic polygon clipping library according to the reference: Alan Murta, “A Generic Polygon Clipping Library”, Department of Computer Science, University of Manchester, 1999, the contents of which are hereby incorporated by reference thereto, is used. This mock layout is used to calculate the various area calculations used in the evaluations.
- The evaluation of white space involves a score of the white space, which is a measure of the amount of white space on the page and is determined by calculating the percentage of the page area not filled by any images. A score of 0.0 indicates a layout where the images placed on the page take up the entire page area. A score of 1.0 is indicative of an empty page with no images on it (i.e. the entire page is white space). As indicated in Table 3, there are system parameters that limit the acceptable amount of white space allowed. Solutions that result in white space percentages below the minimum or greater than the maximum are tagged as unacceptable solutions and given a final fitness value of 0.0.
- In evaluating overlap, the maximum overlap between any two images is considered. The overlap score is defined as the maximum percentage of any image area covered by another image over all of the images placed on the page. Similar to white space, there are system parameters that limit the acceptable amount of overlap allowed. See Table 3, above. Solutions that result in overlap scores below the minimum or greater than the maximum are tagged as unacceptable solutions and given a final fitness value of 0.0.
- The rotation score is an indication of the total amount of image rotation on an album page. It is derived by averaging the absolute values of the image rotations over all of the images. This average is scaled by θmax to achieve a score between 0.0 and 1.0. Note that since the interpretation of the genome encoding for rotation ensures that the rotation for each image will be between −θmax and +θmax, this rotation evaluation score will never exceed 1.0.
- Spatial balance is evaluated by comparing the image areas in the four quadrants of the album page. The page is first split horizontally into equal halves. The amount of page area containing images for both halves are determined and the ratio of the half with the smaller image area over the half with the larger image area is calculated. For a spatially balanced layout, this ratio will be close to 1.0. The same procedure is performed in the vertical direction. The final spatial balance score is the average of the two ratios.
- The score for rotational balance is calculated in a manner similar to that of rotation. Like with the rotation evaluation, the rotation balance values over all the images are averaged. However, unlike the evaluation for rotation, the actual rotation values and not the absolute values of these rotations are considered when calculating the average. The rational behind this evaluation is that for a rotationally balanced layout, the summation of all the rotations should amount to 0.0. The rotational balance score is obtained by scaling the calculated signed average by θmax and then subtracting this value from 1.0. This way, a very rotationally balanced layout, (i.e. one where the signed average of the rotations is 0.0) will produce a score of 1.0. Similarly, a very rotationally unbalanced layout, one where the average of the rotations is close to θmax will earn a score of 0.0.
- The border symmetry score evaluates how closely the edges of the image on the page form a natural boundary. For each edge of the album page (top, bottom, left, right), a border distance is determined by considering the image endpoint closest to the page edge and calculating the distance from the edge to the endpoint. The mean distance amongst the four edges is determined and the average distance from the mean amongst the four edges is calculated. This average is scaled by BDRmax and subtracted from 1.0 to generate the final score. Note that if the average distance is greater than BDRmax, the solution is flagged as unfit and is assigned a final fitness value of 0.0.
- The emphasis score measures the proportionality of the size of the images with respect to the emphasis values assigned to the images. The rationale behind this evaluation stems from the notion that images with large emphasis values should take up more space on the page. The comparison made during evaluation is relative to the sizes of all of the images. For each image, the size relative to the largest image in the group is calculated and expressed as a percentage. This percentage is then subtracted from the emphasis value assigned to the image. Then mean difference amongst all the images on the page is calculated and this average is subtracted from 1.0, assuring that an emphasis score of 1.0 indicates a strong positive correlation between image size and emphasis values.
- In a fashion similar to the approach used with the Page Creator Module, in determining the final fitness of the solution, both the preferences and the importance of each criterion, as specified by the preference and importance parameters supplied by the user, must also be considered. Thus, the Image Placement Module determines final fitness in the same manner as the page creator module described herein before. The only difference is in the set of criteria considered in performing the evaluation. Thus, final fitness for the Image Placement Module according to an illustrative embodiment of the present invention is described by:
- where I represents the set of importance parameters value (1 per criteria), P represents the set of preference parameter values (1 per criteria) and E represents the set of raw evaluations scores as determined by the procedures outlined above (1 per criteria).
- To gain a better understanding of the Image Placement Module layout functionality, it is beneficial to contemplate a series of exemplar page layouts, each of which exercises certain aspects of this module. These layouts appear in
FIGS. 23 through 38 . In each of the layouts, the illustrative images presented inFIG. 19 (reference numeral 418),FIG. 20 (reference numeral 420),FIG. 21 (reference numeral 422), andFIG. 22 (reference numeral 424) are supplied as input to the Image Placement Module). Note that the values given below each image are the emphasis values for the particular image. In the following example layouts, each criterion will be considered separately to illustrate the effect of manipulating the preference for the criteria in question. In each of the examples, the module ran the genetic algorithm through three hundred generations resulting in layouts with final scores ranging from 0.85-0.95. -
FIGS. 23 and 24 illustrate the effect of the white space preference. Note that SPmax has been set to 75% and SPmin has been set to 5%. ForFIG. 23 ,image 426, the preference is set to 0.0, and, forFIG. 24 ,image 428, the preference is set to 1.0. -
FIGS. 25 and 26 illustrate the effect of overlap. In this example, the white space preference is kept constant between the two layouts. The value of OLmin is set to be 0% whereas the value of OLmax is set at 25%. ForFIG. 25 ,image 430, the preference is set to 0.0, and, forFIG. 26 ,image 432, the preference is set to 1.0. - The effect of rotation is illustrated in
FIGS. 27 and 28 . Once again, the white space preference is kept constant between the two layouts. In addition, the overlap preference is set to 0. The value of θmax is set at 30 degrees. ForFIG. 27 ,image 434, the preference is set to 0.0, and, forFIG. 28 , image 4436, the preference is set to 1.0. - In
FIGS. 29 and 30 , the effects of rotational balance are shown. For sake of illustration, the rotation preference for both layouts is set to 1.0. ForFIG. 29 ,image 438, the preference is set to 0.0, and, forFIG. 30 ,image 440, the preference is set to 1.0. - The effects of spatial balance are shown in
FIGS. 31 and 32 . Note that setting the preference to 0.0 results in a layout where one quadrant of the page is completely empty where the rest have images in them. ForFIG. 31 ,image 442, the preference is set to 0.0, and, forFIG. 32 ,image 444, the preference is set to 1.0. -
FIGS. 33 and 34 show the effects of border symmetry. In order to emphasize the border effect, the rotation and overlap preferences in this example are set to 0.0. ForFIG. 33 ,image 446, the preference is set to 0.0, and, forFIG. 34 ,image 448, the preference is set to 1.0. - Finally, the effects of emphasis are illustrated in
FIGS. 35 and 36 . Note that setting the emphasis preference to 1.0 indicates that the user wants a strong relationship between the size of the image and the emphasis value. Considering the method by which emphasis is evaluated, described herein before, setting this preference set to 0.0, indicates to the system that this relationship should be reversed. This is clearly the effect shown in these Figs. considering the emphasis values listed for the reference images fromFIGS. 19 through 21 . ForFIG. 35 ,image 450, the preference is set to 0.0, and, forFIG. 36 , image 4452, the preference is set to 1.0. - The variety of page layouts that can be produced by the Image Placement Module is best illustrated when modifying several preference parameters at once. In
FIGS. 37 and 38 , the preferences for white space, rotation, and overlap are all modified between the two layouts. The results are the two layouts illustrated with two entirely different feels to them,FIG. 37 ,image 454, being very traditional and rigid,FIG. 38 ,image 456, having more of a whimsical quality to it. - Thus, the present invention has been described herein with reference to a particular embodiment for a particular application. Those having ordinary skill in the art and access to the present teachings will recognize additional modifications, applications and embodiments within the scope thereof. For example,
- It is therefore intended by the appended claims to cover any and all such applications, modifications and embodiments within the scope of the present invention.
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/957,675 US20080094420A1 (en) | 2000-12-29 | 2007-12-17 | System and method for automatic layout of images in digital albums |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/750,858 US7340676B2 (en) | 2000-12-29 | 2000-12-29 | System and method for automatic layout of images in digital albums |
US11/957,675 US20080094420A1 (en) | 2000-12-29 | 2007-12-17 | System and method for automatic layout of images in digital albums |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/750,858 Continuation US7340676B2 (en) | 2000-12-29 | 2000-12-29 | System and method for automatic layout of images in digital albums |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080094420A1 true US20080094420A1 (en) | 2008-04-24 |
Family
ID=25019436
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/750,858 Expired - Fee Related US7340676B2 (en) | 2000-12-29 | 2000-12-29 | System and method for automatic layout of images in digital albums |
US11/957,675 Abandoned US20080094420A1 (en) | 2000-12-29 | 2007-12-17 | System and method for automatic layout of images in digital albums |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/750,858 Expired - Fee Related US7340676B2 (en) | 2000-12-29 | 2000-12-29 | System and method for automatic layout of images in digital albums |
Country Status (3)
Country | Link |
---|---|
US (2) | US7340676B2 (en) |
EP (1) | EP1220531A3 (en) |
JP (1) | JP2002288669A (en) |
Cited By (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050071783A1 (en) * | 2003-09-30 | 2005-03-31 | Atkins C. Brian | Automatic photo album page layout |
US20050264847A1 (en) * | 2004-06-01 | 2005-12-01 | Canon Kabushiki Kaisha | Image forming apparatus, image forming method, computer program, and computer-readable storage medium |
US20060220983A1 (en) * | 2005-03-15 | 2006-10-05 | Fuji Photo Film Co., Ltd. | Album creating apparatus, album generating method and program |
US20060279566A1 (en) * | 2005-06-10 | 2006-12-14 | Atkins C B | Constraint-based albuming of graphic elements |
US20070118797A1 (en) * | 2003-08-29 | 2007-05-24 | Paul Layzell | Constrained document layout |
US20090177959A1 (en) * | 2008-01-08 | 2009-07-09 | Deepayan Chakrabarti | Automatic visual segmentation of webpages |
US20100169767A1 (en) * | 2008-09-18 | 2010-07-01 | Canon Kabushiki Kaisha | Method and apparatus for processing layout and storage medium |
US20100199227A1 (en) * | 2009-02-05 | 2010-08-05 | Jun Xiao | Image collage authoring |
US20100289818A1 (en) * | 2009-05-12 | 2010-11-18 | Canon Kabushiki Kaisha | Image layout device, image layout method, and storage medium |
US20110035345A1 (en) * | 2009-08-10 | 2011-02-10 | Yahoo! Inc. | Automatic classification of segmented portions of web pages |
US20110035374A1 (en) * | 2009-08-10 | 2011-02-10 | Yahoo! Inc. | Segment sensitive query matching of documents |
WO2011053287A1 (en) * | 2009-10-28 | 2011-05-05 | Hewlett-Packard Development Company, L.P. | Determining a layout of graphic objects according to a layout density map |
US20110234818A1 (en) * | 2010-03-23 | 2011-09-29 | Nikon Corporation | Image processing device and computer-readable computer program product containing image processing program |
US20110234613A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US20120027293A1 (en) * | 2010-07-27 | 2012-02-02 | Cok Ronald S | Automated multiple image product method |
US20120027311A1 (en) * | 2010-07-27 | 2012-02-02 | Cok Ronald S | Automated image-selection method |
US8275733B2 (en) | 2009-03-11 | 2012-09-25 | Hewlett-Packard Development Company, L.P. | Creating an album |
US8417728B1 (en) * | 2007-08-03 | 2013-04-09 | Adobe Systems Incorporated | User interfaces, methods, and systems for developing computer applications using artwork |
US20130176333A1 (en) * | 2012-01-11 | 2013-07-11 | Research In Motion Limited | Interface for previewing image content |
US8584015B2 (en) | 2010-10-19 | 2013-11-12 | Apple Inc. | Presenting media content items using geographical data |
US20140009796A1 (en) * | 2012-07-09 | 2014-01-09 | Canon Kabushiki Kaisha | Information processing apparatus and control method thereof |
US20140108922A1 (en) * | 2012-10-16 | 2014-04-17 | Theo Gravity | Content layout determination |
US20140233811A1 (en) * | 2012-05-15 | 2014-08-21 | Google Inc. | Summarizing a photo album |
US20150189107A1 (en) * | 2012-09-03 | 2015-07-02 | Sony Corporation | Information processing device, information processing method, and program |
CN105183478A (en) * | 2015-09-11 | 2015-12-23 | 中山大学 | Webpage reestablishing method and device based on color transmission |
WO2016002985A1 (en) * | 2014-07-01 | 2016-01-07 | Lg Electronics Inc. | Electronic device and method for controlling the same |
US9311530B1 (en) | 2012-05-15 | 2016-04-12 | Google Inc. | Summarizing a photo album in a social network system |
US9348801B2 (en) | 2010-08-10 | 2016-05-24 | HP Brasil LTDA | System and method for automatic page layout |
US9594730B2 (en) | 2010-07-01 | 2017-03-14 | Yahoo! Inc. | Annotating HTML segments with functional labels |
US9858295B2 (en) | 2014-06-24 | 2018-01-02 | Google Llc | Ranking and selecting images for display from a set of images |
US10013395B2 (en) | 2012-07-09 | 2018-07-03 | Canon Kabushiki Kaisha | Apparatus, control method thereof, and storage medium that determine a layout image from a generated plurality of layout images by evaluating selected target images |
JP2022058399A (en) * | 2016-01-05 | 2022-04-12 | エボルブ テクノロジー ソリューションズ インク. | Machine learning based web interface generation and testing system |
Families Citing this family (196)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050210413A1 (en) * | 1999-11-24 | 2005-09-22 | Quek Su M | Utilization of digital images |
US7054508B2 (en) * | 2000-08-03 | 2006-05-30 | Canon Kabushiki Kaisha | Data editing apparatus and method |
US20020154147A1 (en) * | 2001-04-20 | 2002-10-24 | Battles Amy E. | Photo ranking system for creating digital album pages |
JP4749628B2 (en) * | 2001-09-07 | 2011-08-17 | パナソニック株式会社 | Album creating apparatus, album creating method, and album creating program |
US20030090498A1 (en) * | 2001-11-13 | 2003-05-15 | Photela, Inc. | Method and apparatus for the creation of digital photo albums |
US7050097B2 (en) | 2001-11-13 | 2006-05-23 | Microsoft Corporation | Method and apparatus for the display of still images from image files |
JP2004015467A (en) * | 2002-06-07 | 2004-01-15 | Ricoh Co Ltd | Image forming apparatus |
US7027071B2 (en) * | 2002-07-02 | 2006-04-11 | Hewlett-Packard Development Company, L.P. | Selecting elements from an electronic document |
GB2406254B (en) * | 2002-08-30 | 2006-03-15 | Hewlett Packard Development Co | Page composition |
AU2002952711A0 (en) * | 2002-11-18 | 2002-11-28 | Typefi Systems Pty Ltd | A method of formatting documents |
US7362919B2 (en) * | 2002-12-12 | 2008-04-22 | Eastman Kodak Company | Method for generating customized photo album pages and prints based on people and gender profiles |
US7240047B2 (en) * | 2002-12-23 | 2007-07-03 | Hewlett-Packard Development Company, L.P. | Apparatus and method for market-based document layout selection |
US20040122824A1 (en) * | 2002-12-23 | 2004-06-24 | Clearwater Scott H. | Apparatus and method for market-based document content selection |
US20040122856A1 (en) * | 2002-12-23 | 2004-06-24 | Clearwater Scott H. | Apparatus and method for market-based document content and layout selection |
US20040122805A1 (en) * | 2002-12-23 | 2004-06-24 | Sang Henry W. | Apparatus and method for market-based graphical grouping |
US7472110B2 (en) * | 2003-01-29 | 2008-12-30 | Microsoft Corporation | System and method for employing social networks for information discovery |
US7079293B2 (en) * | 2003-04-08 | 2006-07-18 | Tr Marketing, Inc. | Digital album and methods of use |
US7019871B2 (en) * | 2003-04-08 | 2006-03-28 | Tr Marketing, Inc. | Digital photo album having a built-in scanner |
JP4148011B2 (en) * | 2003-04-25 | 2008-09-10 | セイコーエプソン株式会社 | Print job creation apparatus, print job creation method, and program used therefor |
AU2003902954A0 (en) * | 2003-06-12 | 2003-06-26 | Canon Information Systems Research Australia Pty Ltd | Geometric space decoration in a graphical design system |
GB2404269A (en) * | 2003-07-24 | 2005-01-26 | Hewlett Packard Development Co | Estimating symmetry in a document |
JP2005057671A (en) * | 2003-08-07 | 2005-03-03 | Dainippon Screen Mfg Co Ltd | Document processor, printing system, and method for adding object to document, and program |
US7308650B2 (en) | 2003-08-29 | 2007-12-11 | Seiko Epson Corporation | Image layout device |
JP3633608B2 (en) * | 2003-08-29 | 2005-03-30 | セイコーエプソン株式会社 | Image arrangement apparatus and method |
EP1610542B1 (en) * | 2003-09-10 | 2012-04-11 | Seiko Epson Corporation | Output image data generation device and output image data generation method |
US20050068583A1 (en) * | 2003-09-30 | 2005-03-31 | Gutkowski Lawrence J. | Organizing a digital image |
US20050071479A1 (en) * | 2003-09-30 | 2005-03-31 | Dimitris Achlioptas | Smart button |
US8056013B2 (en) * | 2005-05-13 | 2011-11-08 | Hewlett-Packard Development Company, L.P. | Method for arranging graphic assemblies |
US8065627B2 (en) * | 2003-09-30 | 2011-11-22 | Hewlett-Packard Development Company, L.P. | Single pass automatic photo album page layout |
US7640516B2 (en) * | 2003-09-30 | 2009-12-29 | Hewlett-Packard Development Company, L.P. | Arranging graphic objects on pages |
US7555730B2 (en) * | 2005-05-12 | 2009-06-30 | Hewlett-Packard Development Company, L.P. | Method for arranging graphic assemblies |
US7188310B2 (en) * | 2003-10-09 | 2007-03-06 | Hewlett-Packard Development Company, L.P. | Automatic layout generation for photobooks |
US7478340B2 (en) * | 2003-10-22 | 2009-01-13 | Microsoft Corporation | Systems and methods for managing preparation of graphical elements for presentation |
US20050091594A1 (en) * | 2003-10-23 | 2005-04-28 | Microsoft Corporation | Systems and methods for preparing graphical elements for presentation |
US7109848B2 (en) * | 2003-11-17 | 2006-09-19 | Nokia Corporation | Applications and methods for providing a reminder or an alert to a digital media capture device |
US20050105374A1 (en) * | 2003-11-17 | 2005-05-19 | Nokia Corporation | Media diary application for use with digital device |
US8010579B2 (en) | 2003-11-17 | 2011-08-30 | Nokia Corporation | Bookmarking and annotating in a media diary application |
US8990255B2 (en) | 2003-11-17 | 2015-03-24 | Nokia Corporation | Time bar navigation in a media diary application |
US20050108643A1 (en) * | 2003-11-17 | 2005-05-19 | Nokia Corporation | Topographic presentation of media files in a media diary application |
US20050123886A1 (en) * | 2003-11-26 | 2005-06-09 | Xian-Sheng Hua | Systems and methods for personalized karaoke |
US7689909B1 (en) * | 2003-11-26 | 2010-03-30 | Szuszczewicz Edward P | System and method for generating photo collages and photo album pages |
US7774718B2 (en) * | 2003-12-17 | 2010-08-10 | Nokia Corporation | Time handle in a media diary application for accessing media files |
JP4125249B2 (en) * | 2004-01-30 | 2008-07-30 | キヤノン株式会社 | Information processing method, information processing apparatus, and computer-readable program |
US7457467B2 (en) | 2004-01-30 | 2008-11-25 | Xerox Corporation | Method and apparatus for automatically combining a digital image with text data |
US20050187943A1 (en) * | 2004-02-09 | 2005-08-25 | Nokia Corporation | Representation of media items in a media file management application for use with a digital device |
US20050235293A1 (en) * | 2004-04-14 | 2005-10-20 | Microsoft Corporation | Methods and systems for framework layout editing operations |
EP1589444A3 (en) * | 2004-04-21 | 2008-03-12 | Samsung Electronics Co., Ltd. | Method, medium, and apparatus for detecting situation change of digital photos and method, medium, and apparatus for situation-based photo clustering in digital photo album |
US7296224B2 (en) * | 2004-04-23 | 2007-11-13 | Hewlett-Packard Development Company, L.P. | Method for assigning graphical images to pages |
US9235655B2 (en) * | 2004-05-21 | 2016-01-12 | Hewlett-Packard Development Company, L.P. | Task-based design evaluation |
US8719699B2 (en) | 2004-05-26 | 2014-05-06 | Typefi Systems Pty. Ltd. | Dynamic positioning of components using document templates |
US8657606B2 (en) * | 2004-07-02 | 2014-02-25 | Paul Fisher | Asynchronous art jurying system |
WO2006012685A1 (en) * | 2004-08-04 | 2006-02-09 | Momento Pro Pty Ltd | Image layout method |
GB0418444D0 (en) * | 2004-08-19 | 2004-09-22 | Eastman Kodak Co | Montage method |
JP2006074592A (en) * | 2004-09-03 | 2006-03-16 | Canon Inc | Electronic album edit apparatus, control method thereof, program thereof, and computer readable storage medium with program stored |
US7752548B2 (en) * | 2004-10-29 | 2010-07-06 | Microsoft Corporation | Features such as titles, transitions, and/or effects which vary according to positions |
US7656543B2 (en) * | 2004-11-12 | 2010-02-02 | Hewlett-Packard Development Company, L.P. | Albuming images |
US7609847B2 (en) * | 2004-11-23 | 2009-10-27 | Hewlett-Packard Development Company, L.P. | Methods and systems for determining object layouts |
EP1830320A4 (en) * | 2004-12-24 | 2010-10-20 | Nat Univ Corp Yokohama Nat Uni | Image processor |
US8024658B1 (en) * | 2005-01-09 | 2011-09-20 | Apple Inc. | Application for designing photo albums |
WO2006079991A2 (en) * | 2005-01-28 | 2006-08-03 | Koninklijke Philips Electronics, N.V. | Dynamic photo collage |
US8161377B2 (en) * | 2005-03-01 | 2012-04-17 | Hewlett-Packard Development Company, L.P. | Arranging images on pages of an album |
JP2006253992A (en) * | 2005-03-10 | 2006-09-21 | Fuji Photo Film Co Ltd | Image layout apparatus and method, and program |
JP2006295890A (en) | 2005-03-15 | 2006-10-26 | Fuji Photo Film Co Ltd | Album creating apparatus, album creating method and program |
JP2006293986A (en) | 2005-03-15 | 2006-10-26 | Fuji Photo Film Co Ltd | Album generating apparatus, album generation method and program |
JP2006293985A (en) | 2005-03-15 | 2006-10-26 | Fuji Photo Film Co Ltd | Program, apparatus and method for producing album |
US20060209214A1 (en) * | 2005-03-17 | 2006-09-21 | Xerox Corporation | Digital photo album systems and methods |
US20060248071A1 (en) * | 2005-04-28 | 2006-11-02 | Xerox Corporation | Automated document localization and layout method |
JP2006331393A (en) * | 2005-04-28 | 2006-12-07 | Fujifilm Holdings Corp | Album creating apparatus, album creating method and program |
US7595809B2 (en) | 2005-05-11 | 2009-09-29 | Hewlett-Packard Development Company, L.P. | Method and system for determining an occlusion cost for concurrently presenting one or more images in a shared viewing region |
US7760956B2 (en) | 2005-05-12 | 2010-07-20 | Hewlett-Packard Development Company, L.P. | System and method for producing a page using frames of a video stream |
US7711211B2 (en) * | 2005-06-08 | 2010-05-04 | Xerox Corporation | Method for assembling a collection of digital images |
US7474801B2 (en) * | 2005-07-07 | 2009-01-06 | Shutterfly, Inc. | Automatic generation of a photo album |
KR100696102B1 (en) * | 2005-07-21 | 2007-03-20 | 삼성전자주식회사 | Image forming apparatus and image forming method |
US20070028236A1 (en) * | 2005-07-27 | 2007-02-01 | Typefi System Pty Ltd. | File transfer system |
KR100727961B1 (en) * | 2005-07-29 | 2007-06-13 | 삼성전자주식회사 | N-up display method and apparatus, image forming device thereof |
US7870503B1 (en) * | 2005-08-30 | 2011-01-11 | Adobe Systems Incorporated | Technique for analyzing and graphically displaying document order |
JP2007116658A (en) * | 2005-09-26 | 2007-05-10 | Fujifilm Corp | Image assembly creating system, image assembly creating method and image assembly creating program |
US7644364B2 (en) * | 2005-10-14 | 2010-01-05 | Microsoft Corporation | Photo and video collage effects |
US8438476B2 (en) * | 2005-10-14 | 2013-05-07 | Uhlig Llc | Dynamic variable-content publishing |
KR100703704B1 (en) | 2005-11-02 | 2007-04-06 | 삼성전자주식회사 | Apparatus and method for creating dynamic moving image automatically |
US9354766B2 (en) * | 2006-01-27 | 2016-05-31 | Cimpress Schweiz Gmbh | Automated composite image design |
US7864978B2 (en) * | 2006-02-06 | 2011-01-04 | Microsoft Corporation | Smart arrangement and cropping for photo views |
US8504932B2 (en) * | 2006-04-13 | 2013-08-06 | Shutterfly, Inc. | Image collage builder |
US7848596B2 (en) | 2006-05-24 | 2010-12-07 | Hewlett-Packard Development Company, L.P. | Templated collage generation with occlusion costing |
US7668400B2 (en) | 2006-07-31 | 2010-02-23 | Hewlett-Packard Development Company, L.P. | Image layout constraint generation |
JP4248584B2 (en) * | 2006-07-31 | 2009-04-02 | シャープ株式会社 | Display device, display program, and computer-readable recording medium |
JP4762827B2 (en) * | 2006-08-22 | 2011-08-31 | 富士フイルム株式会社 | Electronic album generation apparatus, electronic album generation method, and program thereof |
US8144919B2 (en) * | 2006-09-22 | 2012-03-27 | Fuji Xerox Co., Ltd. | Annealing algorithm for non-rectangular shaped stained glass collages |
KR100813170B1 (en) * | 2006-09-27 | 2008-03-17 | 삼성전자주식회사 | Method and system for semantic event indexing by analyzing user annotation of digital photos |
US8578267B2 (en) * | 2006-09-28 | 2013-11-05 | Hewlett-Packard Development Company, L.P. | Graphic assembly layout with maximum page coverage and minimum content removal |
US8375302B2 (en) | 2006-11-17 | 2013-02-12 | Microsoft Corporation | Example based video editing |
US10083536B2 (en) * | 2007-01-12 | 2018-09-25 | Autodesk, Inc. | System for mapping animation from a source character to a destination character while conserving angular configuration |
US8078969B2 (en) * | 2007-03-05 | 2011-12-13 | Shutterfly, Inc. | User interface for creating image collage |
US8456691B2 (en) * | 2008-11-21 | 2013-06-04 | Xerox Corporation | Dynamic imposition based on perceived page value |
US20100115399A1 (en) * | 2007-03-15 | 2010-05-06 | Koninklijke Philips Electronics N.V. | Method and apparatus for generating an album of images |
US20080235603A1 (en) * | 2007-03-21 | 2008-09-25 | Holm Aaron H | Digital file management system with dynamic roles assignment and user level image/data interchange |
JP4987538B2 (en) * | 2007-03-29 | 2012-07-25 | 富士フイルム株式会社 | Album creating apparatus, method and program |
JP4928333B2 (en) * | 2007-04-12 | 2012-05-09 | キヤノン株式会社 | Content display device and content display method |
US9239835B1 (en) * | 2007-04-24 | 2016-01-19 | Wal-Mart Stores, Inc. | Providing information to modules |
US9529438B2 (en) | 2007-06-27 | 2016-12-27 | Hewlett-Packard Development Company, L.P. | Printing structured documents |
JP5114114B2 (en) * | 2007-07-03 | 2013-01-09 | 京セラドキュメントソリューションズ株式会社 | Image forming apparatus |
US8225208B2 (en) * | 2007-08-06 | 2012-07-17 | Apple Inc. | Interactive frames for images and videos displayed in a presentation application |
US8120810B2 (en) * | 2007-08-10 | 2012-02-21 | International Business Machines Corporation | Method and system for adjusting scanned images |
US8289562B2 (en) * | 2007-09-21 | 2012-10-16 | Fujifilm Corporation | Image processing apparatus, method and recording medium |
US8862986B2 (en) * | 2007-10-01 | 2014-10-14 | Hewlett-Packard Development Company, L.P. | Arranging graphic objects on a page with relative position based control |
US8122356B2 (en) * | 2007-10-03 | 2012-02-21 | Eastman Kodak Company | Method for image animation using image value rules |
WO2009136888A1 (en) | 2008-05-06 | 2009-11-12 | Hewlett-Packard Development Company, L.P. | Spatiotemporal media object layouts |
US8042039B2 (en) * | 2008-05-25 | 2011-10-18 | Hewlett-Packard Development Company, L.P. | Populating a dynamic page template with digital content objects according to constraints specified in the dynamic page template |
US10282391B2 (en) | 2008-07-03 | 2019-05-07 | Ebay Inc. | Position editing tool of collage multi-media |
US8893015B2 (en) | 2008-07-03 | 2014-11-18 | Ebay Inc. | Multi-directional and variable speed navigation of collage multi-media |
US8365092B2 (en) | 2008-07-03 | 2013-01-29 | Ebay Inc. | On-demand loading of media in a multi-media presentation |
US8131114B2 (en) * | 2008-09-22 | 2012-03-06 | Shutterfly, Inc. | Smart photobook creation |
US8547588B2 (en) * | 2009-01-23 | 2013-10-01 | Hewlett-Packard Development Company, L.P. | Method and article for providing layout flexibility |
US8161384B2 (en) * | 2009-04-23 | 2012-04-17 | Hewlett-Packard Development Company, L.P. | Arranging graphic objects on a page with text |
US8291314B2 (en) * | 2009-04-23 | 2012-10-16 | Hewlett-Packard Development Company, L.P. | Arranging graphic objects on a page |
WO2010151255A1 (en) * | 2009-06-24 | 2010-12-29 | Hewlett-Packard Development Company, L.P. | Image album creation |
US20110004839A1 (en) * | 2009-07-02 | 2011-01-06 | Derek Cha | User-customized computer display method |
US8806331B2 (en) * | 2009-07-20 | 2014-08-12 | Interactive Memories, Inc. | System and methods for creating and editing photo-based projects on a digital network |
US8854395B2 (en) * | 2009-07-30 | 2014-10-07 | Intellectual Ventures Fund 83 Llc | Method for producing artistic image template designs |
US8538986B2 (en) * | 2009-07-30 | 2013-09-17 | Intellectual Ventures Fund 83 Llc | System for coordinating user images in an artistic design |
US8289340B2 (en) * | 2009-07-30 | 2012-10-16 | Eastman Kodak Company | Method of making an artistic digital template for image display |
US8422794B2 (en) | 2009-07-30 | 2013-04-16 | Intellectual Ventures Fund 83 Llc | System for matching artistic attributes of secondary image and template to a primary image |
US20110029860A1 (en) * | 2009-07-30 | 2011-02-03 | Ptucha Raymond W | Artistic digital template for image display |
US8237819B2 (en) * | 2009-07-30 | 2012-08-07 | Eastman Kodak Company | Image capture method with artistic template design |
US8332427B2 (en) * | 2009-07-30 | 2012-12-11 | Eastman Kodak Company | Method of generating artistic template designs |
US8849853B2 (en) * | 2009-07-30 | 2014-09-30 | Intellectual Ventures Fund 83 Llc | Method for matching artistic attributes of a template and secondary images to a primary image |
US20110029635A1 (en) * | 2009-07-30 | 2011-02-03 | Shkurko Eugene I | Image capture device with artistic template design |
US8274523B2 (en) * | 2009-07-30 | 2012-09-25 | Eastman Kodak Company | Processing digital templates for image display |
US8212834B2 (en) * | 2009-07-30 | 2012-07-03 | Eastman Kodak Company | Artistic digital template for image display |
US20110029914A1 (en) * | 2009-07-30 | 2011-02-03 | Whitby Laura R | Apparatus for generating artistic image template designs |
US20110029562A1 (en) * | 2009-07-30 | 2011-02-03 | Whitby Laura R | Coordinating user images in an artistic design |
US8345057B2 (en) * | 2009-07-30 | 2013-01-01 | Eastman Kodak Company | Context coordination for an artistic digital template for image display |
JP5366699B2 (en) * | 2009-08-10 | 2013-12-11 | キヤノン株式会社 | Image processing apparatus, image processing method, and image processing program |
WO2011031257A1 (en) * | 2009-09-09 | 2011-03-17 | Hewlett-Packard Development Company, L.P. | Generating graphic object collages |
US9383916B2 (en) | 2009-09-30 | 2016-07-05 | Microsoft Technology Licensing, Llc | Dynamic image presentation |
US8644334B2 (en) | 2009-09-30 | 2014-02-04 | Silicon Image, Inc. | Messaging to provide data link integrity |
WO2011065949A1 (en) * | 2009-11-25 | 2011-06-03 | Hewlett-Packard Development Company, Lp | Automatic page layout system and method |
JP2011124662A (en) * | 2009-12-08 | 2011-06-23 | Canon Inc | Image processing apparatus, image processing method, and computer program |
US9319640B2 (en) * | 2009-12-29 | 2016-04-19 | Kodak Alaris Inc. | Camera and display system interactivity |
US20110157218A1 (en) * | 2009-12-29 | 2011-06-30 | Ptucha Raymond W | Method for interactive display |
US9253447B2 (en) * | 2009-12-29 | 2016-02-02 | Kodak Alaris Inc. | Method for group interactivity |
US8390648B2 (en) * | 2009-12-29 | 2013-03-05 | Eastman Kodak Company | Display system for personalized consumer goods |
US9179102B2 (en) * | 2009-12-29 | 2015-11-03 | Kodak Alaris Inc. | Group display system |
US20110211753A1 (en) * | 2010-03-01 | 2011-09-01 | Kenneth Kun Lee | Automatic processing of pre-selected filters that are associated with each photo location within the template in relation to creating the photobook |
US20110213795A1 (en) * | 2010-03-01 | 2011-09-01 | Kenneth Kun Lee | Automatic creation of alternative layouts using the same selected photos by applying special filters and/or changing photo locations in relation to creating the photobook |
JP5389724B2 (en) * | 2010-03-31 | 2014-01-15 | 富士フイルム株式会社 | Image processing apparatus, image processing method, and program |
US8842934B2 (en) * | 2010-07-20 | 2014-09-23 | Canon Kabushiki Kaisha | Image processing apparatus and method for controlling the image processing apparatus |
US9558191B2 (en) * | 2010-08-31 | 2017-01-31 | Picaboo Corporation | Automatic identification of photo books system and method |
US20120054600A1 (en) * | 2010-08-31 | 2012-03-01 | Picaboo Corporation | Image clustering and page layout system and method |
JP5323103B2 (en) * | 2010-09-03 | 2013-10-23 | 三菱電機株式会社 | Graphical user interface device |
US10127697B2 (en) * | 2010-10-28 | 2018-11-13 | Kodak Alaris Inc. | Imaging product selection system |
US8913301B2 (en) | 2010-10-28 | 2014-12-16 | Intellectual Ventures Fund 83 Llc | Imaging product layout method |
US8922843B2 (en) | 2010-10-28 | 2014-12-30 | Kodak Alaris Inc. | Imaging product layout system |
EP2633428A1 (en) * | 2010-10-30 | 2013-09-04 | Hewlett-Packard Development Company, L.P. | Optimizing hyper parameters of probabilistic model for mixed text-and-graphics layout template |
US8805165B2 (en) * | 2010-11-09 | 2014-08-12 | Kodak Alaris Inc. | Aligning and summarizing different photo streams |
US20120206496A1 (en) * | 2011-02-11 | 2012-08-16 | Cok Ronald S | System for imaging product layout |
US8923629B2 (en) | 2011-04-27 | 2014-12-30 | Hewlett-Packard Development Company, L.P. | System and method for determining co-occurrence groups of images |
USD658670S1 (en) | 2011-05-27 | 2012-05-01 | Microsoft Corporation | Display screen with user interface |
US9946429B2 (en) * | 2011-06-17 | 2018-04-17 | Microsoft Technology Licensing, Llc | Hierarchical, zoomable presentations of media sets |
JP5930653B2 (en) * | 2011-10-14 | 2016-06-08 | キヤノン株式会社 | Information processing apparatus, image selection method, and program |
US9269323B2 (en) | 2011-10-28 | 2016-02-23 | Microsoft Technology Licensing, Llc | Image layout for a display |
US9558577B2 (en) | 2012-02-07 | 2017-01-31 | Rowland Hobbs | Rhythmic mosaic generation methods, apparatuses and media |
US10592196B2 (en) | 2012-02-07 | 2020-03-17 | David H. Sonnenberg | Mosaic generating platform methods, apparatuses and media |
US10127000B2 (en) | 2012-02-07 | 2018-11-13 | Rowland Hobbs | Mosaic generating platform methods, apparatuses and media |
JP5568148B2 (en) * | 2012-03-27 | 2014-08-06 | 富士フイルム株式会社 | Layout device, layout method, and program |
US8990140B2 (en) | 2012-06-08 | 2015-03-24 | Microsoft Technology Licensing, Llc | Transforming data into consumable content |
US9076140B2 (en) | 2012-06-26 | 2015-07-07 | Wal-Mart Stores, Inc. | Image embedding apparatus and method |
JP6045232B2 (en) * | 2012-07-09 | 2016-12-14 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
US9595298B2 (en) | 2012-07-18 | 2017-03-14 | Microsoft Technology Licensing, Llc | Transforming data to create layouts |
US9009092B2 (en) | 2012-07-19 | 2015-04-14 | Microsoft Technology Licensing, Llc | Creating variations when transforming data into consumable content |
US9330437B2 (en) | 2012-09-13 | 2016-05-03 | Blackberry Limited | Method for automatically generating presentation slides containing picture elements |
US8861005B2 (en) * | 2012-09-28 | 2014-10-14 | Interactive Memories, Inc. | Methods for real time discovery, selection, and engagement of most economically feasible printing service vendors among multiple known vendors |
US20140105466A1 (en) * | 2012-10-16 | 2014-04-17 | Ocean Images UK Ltd. | Interactive photography system and method employing facial recognition |
US9245209B2 (en) | 2012-11-21 | 2016-01-26 | Xerox Corporation | Dynamic bleed area definition for printing of multi-dimensional substrates |
US9116924B2 (en) * | 2013-01-14 | 2015-08-25 | Xerox Corporation | System and method for image selection using multivariate time series analysis |
US9460056B2 (en) * | 2013-01-22 | 2016-10-04 | Xerox Corporation | Dynamic image layout for personalized packages |
JP5883815B2 (en) * | 2013-03-11 | 2016-03-15 | 富士フイルム株式会社 | Electronic album apparatus and operation control method thereof |
US20150029346A1 (en) * | 2013-07-23 | 2015-01-29 | Insurance Auto Auctions, Inc. | Photo inspection guide for vehicle auction |
JP6282065B2 (en) * | 2013-09-05 | 2018-02-21 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
US20160203108A1 (en) * | 2013-09-06 | 2016-07-14 | Smugmug, Inc. | Display scaling application |
US10552886B2 (en) | 2013-11-07 | 2020-02-04 | Yearbooker, Inc. | Methods and apparatus for merchandise generation including an image |
US10713219B1 (en) * | 2013-11-07 | 2020-07-14 | Yearbooker, Inc. | Methods and apparatus for dynamic image entries |
JP2017041044A (en) * | 2015-08-19 | 2017-02-23 | カシオ計算機株式会社 | Display controller, display control method, and program |
CN106803234B (en) * | 2015-11-26 | 2020-06-16 | 腾讯科技(深圳)有限公司 | Picture display control method and device in picture editing |
JP2017099616A (en) * | 2015-12-01 | 2017-06-08 | ソニー株式会社 | Surgical control device, surgical control method and program, and surgical system |
US11086418B2 (en) * | 2016-02-04 | 2021-08-10 | Douzen, Inc. | Method and system for providing input to a device |
US9509942B1 (en) | 2016-02-08 | 2016-11-29 | Picaboo Corporation | Automatic content categorizing system and method |
JP6825284B2 (en) * | 2016-09-21 | 2021-02-03 | カシオ計算機株式会社 | Image creation device, image creation method, and program |
US10380228B2 (en) | 2017-02-10 | 2019-08-13 | Microsoft Technology Licensing, Llc | Output generation based on semantic expressions |
US10740539B2 (en) * | 2017-04-17 | 2020-08-11 | Hewlett-Packard Development Company, L.P. | Page structure adjustments |
KR102421856B1 (en) * | 2017-12-20 | 2022-07-18 | 삼성전자주식회사 | Method and apparatus for processing image interaction |
US10600223B2 (en) * | 2017-12-29 | 2020-03-24 | Laurent Francois MARTIN | Methods and systems for determining the structure of a sorted collection list of layouts |
JP2020004270A (en) * | 2018-06-29 | 2020-01-09 | キヤノン株式会社 | Information processing device, control method and program |
JP2020057381A (en) * | 2018-09-28 | 2020-04-09 | 大日本印刷株式会社 | Information processor, information processing method and program |
CN113366537A (en) * | 2019-03-26 | 2021-09-07 | 富士胶片株式会社 | Image processing apparatus, image processing method, and image processing program |
CN110188253B (en) * | 2019-05-30 | 2021-09-14 | 北京字节跳动网络技术有限公司 | Information display method and device and storage medium |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5911146A (en) * | 1996-05-03 | 1999-06-08 | Mitsubishi Electric Information Technology Center America, Inc. (Ita) | Apparatus and method for automatic yellow pages pagination and layout |
US6222947B1 (en) * | 1997-02-19 | 2001-04-24 | Canon Kabushiki Kaisha | Image editing apparatus and method and medium on which programs are recorded |
US20020095439A1 (en) * | 1997-02-20 | 2002-07-18 | Timothy Merrick Long | Method of positioning display images |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5685002A (en) * | 1993-09-29 | 1997-11-04 | Minolta Co., Ltd. | Image processing system capable of generating a multi-picture image |
AU698055B2 (en) * | 1994-07-14 | 1998-10-22 | Johnson-Grace Company | Method and apparatus for compressing images |
US5987171A (en) * | 1994-11-10 | 1999-11-16 | Canon Kabushiki Kaisha | Page analysis system |
US6173286B1 (en) * | 1996-02-29 | 2001-01-09 | Nth Degree Software, Inc. | Computer-implemented optimization of publication layouts |
DE69832544T2 (en) | 1997-02-19 | 2006-08-10 | Canon K.K. | Arrangement and method for image processing and medium with programs stored on it |
US6324545B1 (en) * | 1997-10-15 | 2001-11-27 | Colordesk Ltd. | Personalized photo album |
US6202061B1 (en) * | 1997-10-24 | 2001-03-13 | Pictra, Inc. | Methods and apparatuses for creating a collection of media |
US6571054B1 (en) * | 1997-11-10 | 2003-05-27 | Nippon Telegraph And Telephone Corporation | Method for creating and utilizing electronic image book and recording medium having recorded therein a program for implementing the method |
JPH11250272A (en) | 1998-01-08 | 1999-09-17 | Xerox Corp | Automatic image layout method and system therefor |
-
2000
- 2000-12-29 US US09/750,858 patent/US7340676B2/en not_active Expired - Fee Related
-
2001
- 2001-12-17 EP EP01204944A patent/EP1220531A3/en not_active Withdrawn
- 2001-12-26 JP JP2001394769A patent/JP2002288669A/en active Pending
-
2007
- 2007-12-17 US US11/957,675 patent/US20080094420A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5911146A (en) * | 1996-05-03 | 1999-06-08 | Mitsubishi Electric Information Technology Center America, Inc. (Ita) | Apparatus and method for automatic yellow pages pagination and layout |
US6222947B1 (en) * | 1997-02-19 | 2001-04-24 | Canon Kabushiki Kaisha | Image editing apparatus and method and medium on which programs are recorded |
US20020095439A1 (en) * | 1997-02-20 | 2002-07-18 | Timothy Merrick Long | Method of positioning display images |
Cited By (51)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070118797A1 (en) * | 2003-08-29 | 2007-05-24 | Paul Layzell | Constrained document layout |
US7634723B2 (en) * | 2003-08-29 | 2009-12-15 | Hewlett-Packard Development Company, L.P. | Constrained document layout |
US20050071783A1 (en) * | 2003-09-30 | 2005-03-31 | Atkins C. Brian | Automatic photo album page layout |
US7743322B2 (en) * | 2003-09-30 | 2010-06-22 | Hewlett-Packard Development Company, L.P. | Automatic photo album page layout |
US20050264847A1 (en) * | 2004-06-01 | 2005-12-01 | Canon Kabushiki Kaisha | Image forming apparatus, image forming method, computer program, and computer-readable storage medium |
US8056002B2 (en) * | 2004-06-01 | 2011-11-08 | Canon Kabushiki Kaisha | Image forming apparatus, image forming method, computer program, and computer-readable storage medium |
US8631322B2 (en) * | 2005-03-15 | 2014-01-14 | Fujifilm Corporation | Album creating apparatus facilitating appropriate image allocation, album generating method and program |
US20060220983A1 (en) * | 2005-03-15 | 2006-10-05 | Fuji Photo Film Co., Ltd. | Album creating apparatus, album generating method and program |
US7644356B2 (en) | 2005-06-10 | 2010-01-05 | Hewlett-Packard Development Company, L.P. | Constraint-based albuming of graphic elements |
US20060279566A1 (en) * | 2005-06-10 | 2006-12-14 | Atkins C B | Constraint-based albuming of graphic elements |
US8417728B1 (en) * | 2007-08-03 | 2013-04-09 | Adobe Systems Incorporated | User interfaces, methods, and systems for developing computer applications using artwork |
US20090177959A1 (en) * | 2008-01-08 | 2009-07-09 | Deepayan Chakrabarti | Automatic visual segmentation of webpages |
US8255793B2 (en) * | 2008-01-08 | 2012-08-28 | Yahoo! Inc. | Automatic visual segmentation of webpages |
US20100169767A1 (en) * | 2008-09-18 | 2010-07-01 | Canon Kabushiki Kaisha | Method and apparatus for processing layout and storage medium |
US8375298B2 (en) * | 2008-09-18 | 2013-02-12 | Canon Kabushiki Kaisha | Method and apparatus for processing layout and storage medium |
US9152292B2 (en) * | 2009-02-05 | 2015-10-06 | Hewlett-Packard Development Company, L.P. | Image collage authoring |
US20100199227A1 (en) * | 2009-02-05 | 2010-08-05 | Jun Xiao | Image collage authoring |
US8275733B2 (en) | 2009-03-11 | 2012-09-25 | Hewlett-Packard Development Company, L.P. | Creating an album |
US20100289818A1 (en) * | 2009-05-12 | 2010-11-18 | Canon Kabushiki Kaisha | Image layout device, image layout method, and storage medium |
US8665294B2 (en) * | 2009-05-12 | 2014-03-04 | Canon Kabushiki Kaisha | Image layout device, image layout method, and storage medium |
US9465872B2 (en) | 2009-08-10 | 2016-10-11 | Yahoo! Inc. | Segment sensitive query matching |
US8849725B2 (en) | 2009-08-10 | 2014-09-30 | Yahoo! Inc. | Automatic classification of segmented portions of web pages |
US20110035345A1 (en) * | 2009-08-10 | 2011-02-10 | Yahoo! Inc. | Automatic classification of segmented portions of web pages |
US9514216B2 (en) | 2009-08-10 | 2016-12-06 | Yahoo! Inc. | Automatic classification of segmented portions of web pages |
US20110035374A1 (en) * | 2009-08-10 | 2011-02-10 | Yahoo! Inc. | Segment sensitive query matching of documents |
US9024965B2 (en) | 2009-10-28 | 2015-05-05 | Hewlett-Packard Development Company, L.P. | Determining a layout of graphic objects according to a layout density map |
WO2011053287A1 (en) * | 2009-10-28 | 2011-05-05 | Hewlett-Packard Development Company, L.P. | Determining a layout of graphic objects according to a layout density map |
US8928768B2 (en) * | 2010-03-23 | 2015-01-06 | Nikon Corporation | Image processing device and computer-readable computer program product containing image processing program |
US20110234818A1 (en) * | 2010-03-23 | 2011-09-29 | Nikon Corporation | Image processing device and computer-readable computer program product containing image processing program |
US20110234613A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US8988456B2 (en) * | 2010-03-25 | 2015-03-24 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US9594730B2 (en) | 2010-07-01 | 2017-03-14 | Yahoo! Inc. | Annotating HTML segments with functional labels |
US20120027293A1 (en) * | 2010-07-27 | 2012-02-02 | Cok Ronald S | Automated multiple image product method |
US20120027311A1 (en) * | 2010-07-27 | 2012-02-02 | Cok Ronald S | Automated image-selection method |
US9348801B2 (en) | 2010-08-10 | 2016-05-24 | HP Brasil LTDA | System and method for automatic page layout |
US8584015B2 (en) | 2010-10-19 | 2013-11-12 | Apple Inc. | Presenting media content items using geographical data |
US8963953B2 (en) * | 2012-01-11 | 2015-02-24 | Blackberry Limited | Interface for previewing image content |
US20130176333A1 (en) * | 2012-01-11 | 2013-07-11 | Research In Motion Limited | Interface for previewing image content |
US9311530B1 (en) | 2012-05-15 | 2016-04-12 | Google Inc. | Summarizing a photo album in a social network system |
US20140233811A1 (en) * | 2012-05-15 | 2014-08-21 | Google Inc. | Summarizing a photo album |
US20140009796A1 (en) * | 2012-07-09 | 2014-01-09 | Canon Kabushiki Kaisha | Information processing apparatus and control method thereof |
US10013395B2 (en) | 2012-07-09 | 2018-07-03 | Canon Kabushiki Kaisha | Apparatus, control method thereof, and storage medium that determine a layout image from a generated plurality of layout images by evaluating selected target images |
US20150189107A1 (en) * | 2012-09-03 | 2015-07-02 | Sony Corporation | Information processing device, information processing method, and program |
US9813566B2 (en) * | 2012-09-03 | 2017-11-07 | Sony Corporation | Information processing device, information processing method, and program for generating a layout |
US20140108922A1 (en) * | 2012-10-16 | 2014-04-17 | Theo Gravity | Content layout determination |
US9858295B2 (en) | 2014-06-24 | 2018-01-02 | Google Llc | Ranking and selecting images for display from a set of images |
US10417277B2 (en) | 2014-06-24 | 2019-09-17 | Google Llc | Ranking and selecting images for display from a set of images |
WO2016002985A1 (en) * | 2014-07-01 | 2016-01-07 | Lg Electronics Inc. | Electronic device and method for controlling the same |
CN105183478A (en) * | 2015-09-11 | 2015-12-23 | 中山大学 | Webpage reestablishing method and device based on color transmission |
JP2022058399A (en) * | 2016-01-05 | 2022-04-12 | エボルブ テクノロジー ソリューションズ インク. | Machine learning based web interface generation and testing system |
JP7394106B2 (en) | 2016-01-05 | 2023-12-07 | エボルブ テクノロジー ソリューションズ インク. | Web interface generation and testing system based on machine learning |
Also Published As
Publication number | Publication date |
---|---|
JP2002288669A (en) | 2002-10-04 |
US7340676B2 (en) | 2008-03-04 |
EP1220531A2 (en) | 2002-07-03 |
US20020122067A1 (en) | 2002-09-05 |
EP1220531A3 (en) | 2005-11-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7340676B2 (en) | System and method for automatic layout of images in digital albums | |
US10387776B2 (en) | Recurrent neural network architectures which provide text describing images | |
Geigel et al. | Automatic page layout using genetic algorithms for electronic albuming | |
Cole et al. | Where do people draw lines? | |
Geigel et al. | Using genetic algorithms for album page layouts | |
US10679041B2 (en) | Hybrid deep learning method for recognizing facial expressions | |
Chu et al. | Optimized comics-based storytelling for temporal image sequences | |
US11308313B2 (en) | Hybrid deep learning method for recognizing facial expressions | |
Niu et al. | Image retargeting quality assessment based on registration confidence measure and noticeability-based pooling | |
Nóbrega et al. | Interactive 3D content insertion in images for multimedia applications | |
Redi et al. | Where is the beauty? retrieving appealing VideoScenes by learning Flickr-based graded judgments | |
Oishi et al. | Retinex-based relighting for night photography | |
Das et al. | Automatic face-based image grouping for albuming | |
Le et al. | Image aesthetic assessment based on image classification and region segmentation | |
US20020158972A1 (en) | Method of image processing an animated figure | |
Comino Trinidad et al. | Easy authoring of image-supported short stories for 3d scanned cultural heritage | |
Kandemir et al. | Beyond saliency: Assessing visual balance with high-level cues | |
Xu et al. | Discernible image mosaic with edge-aware adaptive tiles | |
CN112347742A (en) | Method for generating document image set based on deep learning | |
Segovia et al. | Aesthetic role of transparency and layering in the creation of photo layouts | |
Isaac | Web-based 3D HSV color data gathering and visualization for applied cinemetrics: A case study in Hayao Miyazaki’s animated motion pictures | |
Sobolevskaya et al. | Multimedia Objects Representation in the Digital Knowledge Space | |
Kobayashi et al. | VR Training System to Help Improve Photography Skills | |
Nguyen et al. | PoB | |
Chia et al. | Identification of Panoramic Photographic Image Composition Using Fuzzy Rules |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: EASTMAN KODAK COMPANY, NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GEIGEL, JOSEPH M.;LOUI, ALEXANDER C.;REEL/FRAME:020256/0253;SIGNING DATES FROM 20010131 TO 20010220 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: KODAK (NEAR EAST), INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: FAR EAST DEVELOPMENT LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: QUALEX INC., NORTH CAROLINA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: PAKON, INC., INDIANA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: EASTMAN KODAK COMPANY, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: NPEC INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK PHILIPPINES, LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK AMERICAS, LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK REALTY, INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: CREO MANUFACTURING AMERICA LLC, WYOMING Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: EASTMAN KODAK INTERNATIONAL CAPITAL COMPANY, INC., Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK PORTUGUESA LIMITED, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: LASER-PACIFIC MEDIA CORPORATION, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK AVIATION LEASING LLC, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: FPC INC., CALIFORNIA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK IMAGING NETWORK, INC., CALIFORNIA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 |
|
AS | Assignment |
Owner name: MONUMENT PEAK VENTURES, LLC, TEXAS Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:INTELLECTUAL VENTURES FUND 83 LLC;REEL/FRAME:064599/0304 Effective date: 20230728 |