WO2017130009A1 - Method and apparatus for fluorescence analysis of a sample - Google Patents

Method and apparatus for fluorescence analysis of a sample Download PDF

Info

Publication number
WO2017130009A1
WO2017130009A1 PCT/HU2016/000078 HU2016000078W WO2017130009A1 WO 2017130009 A1 WO2017130009 A1 WO 2017130009A1 HU 2016000078 W HU2016000078 W HU 2016000078W WO 2017130009 A1 WO2017130009 A1 WO 2017130009A1
Authority
WO
WIPO (PCT)
Prior art keywords
fluorescent
holographic
fluorescence
image
detector
Prior art date
Application number
PCT/HU2016/000078
Other languages
French (fr)
Inventor
László ORZÓ
Ákos ZARÁNDY
Balázs WITTNER
Domonkos GERGELYI
Márton Zsolt KISS
Original Assignee
Mta Számitástechnikai És Automatizálási Kutató Intézet
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mta Számitástechnikai És Automatizálási Kutató Intézet filed Critical Mta Számitástechnikai És Automatizálási Kutató Intézet
Publication of WO2017130009A1 publication Critical patent/WO2017130009A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00Investigating characteristics of particles; Investigating permeability, pore-volume, or surface-area of porous materials
    • G01N15/10Investigating individual particles
    • G01N15/14Electro-optical investigation, e.g. flow cytometers
    • G01N15/1434Electro-optical investigation, e.g. flow cytometers using an analyser being characterised by its optical arrangement
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N21/6456Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458Fluorescence microscopy
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/41Refractivity; Phase-affecting properties, e.g. optical path length
    • G01N21/45Refractivity; Phase-affecting properties, e.g. optical path length using interferometric methods; using Schlieren methods
    • G01N21/453Holographic interferometry
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/16Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04Processes or apparatus for producing holograms
    • G03H1/0443Digital holography, i.e. recording holograms with digital recording means
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N2021/6417Spectrofluorimetric devices
    • G01N2021/6419Excitation at two or more wavelengths
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/47Scattering, i.e. diffuse reflection
    • G01N21/4788Diffraction
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04Processes or apparatus for producing holograms
    • G03H1/0443Digital holography, i.e. recording holograms with digital recording means
    • G03H2001/0447In-line recording arrangement

Definitions

  • the invention relates to a method and an apparatus for fluorescence analysis of a sample.
  • the sample analysed according to the invention is preferably translucent and is in a liquid state, and the invention is adapted for identifying fluorescent objects in the sample.
  • the detection of their shape or three-dimensional morphology can also be of significance; this is achieved by analysing the liquid samples comprising the fluorescent object also by holographic means.
  • the primary object of the invention is to provide a method and an apparatus which is free from disadvantages of prior art approaches to the greatest possible extent.
  • a further object of the invention is to provide a method and an apparatus that are capable of generating holographic images of fluorescent objects with sufficient resolution and of differentiating the fluorescent objects from non-fluorescent ones.
  • a further object of the invention is to provide a method and an apparatus wherein, in addition to fulfilling the above objects, the resource demand of holographic reconstruction can be significantly reduced compared to known approaches.
  • An object of the invention is to provide a method and an apparatus wherein the parameters of the applied fluorescence detector and holographic detector can be chosen independently of each other in order to fulfil the requirements for such devices.
  • Fig. 1 is a block diagram illustrating an embodiment of the apparatus according to the invention
  • Figs. 2A-2C are schematic diagrams illustrating the schedule for recording images in an embodiment of the method according to the invention
  • Fig. 3A is an exemplary fluorescence image
  • Fig. 3B is a holographic image corresponding to Fig. 3A
  • Fig. 3C is the reconstruction of the boxed detail of Fig. 3B.
  • Fig. 4 is another exemplary fluorescence image.
  • the method according to the invention is adapted for fluorescence analysis of a sample.
  • the method according to the invention can be particularly preferably applied for analysing liquid, transparent samples.
  • fluorescent objects are typically carried by water as a liquid, transparent medium.
  • the fluorescent objects When subjected to suitable excitation, the fluorescent objects emit fluorescent light, i.e. they give a fluorescent response.
  • the fluorescent response has a wavelength that is different from the wavelength of excitation light, the response typically having a longer wavelength than the excitation.
  • the sample analysed applying the method according to the invention is passed through the sample holder in a flow-through manner, at a suitable (suitably low) speed.
  • the method according to the invention is of course suitable for analysing stationary samples.
  • the method according to the invention is also suitable for analysing solid samples that are sufficiently thin for the fluorescent objects comprised therein to be excited and that the light emitted by them can be detected.
  • a holographic image that is required for carrying out the method can be typically recorded (taken) from such thin samples.
  • Applying sufficiently short exposure times the method and apparatus according to the invention are also suitable for analysing gaseous samples.
  • the method according to the invention may preferably be carried out applying the apparatus according to the invention, an embodiment of which is illustrated in Fig. 1.
  • a specific analysed sample volume of the sample i.e. the sample volume currently analysed
  • a fluorescence image is recorded by imaging (projecting) the fluorescent emitted light of the specific analysed sample volume emitted under the effect of the fluorescent excitation light.
  • the fluorescent emitted light can be imaged directly (without an imaging element) onto the plane of the detector, or - as it will be shown later on - preferably applying an imaging lens.
  • the specific analysed sample volume is that portion of the sample which is located inside a sample holder spatial region 10 during the excitation.
  • fluorescence image is used herein to refer to an image recorded by imaging fluorescent emitted light emitted by fluorescent objects onto a detector, i.e. the image is adapted for displaying the imaging of light originating from a fluorescent source.
  • holographic image is used to refer to an image carrying holographic information; i.e. the holographic image comprises information that can be reconstructed from the recorded hologram.
  • a holographic image of the specific analysed sample volume is recorded overlapping in time with recording the fluorescence image.
  • both a holographic image and a fluorescence image of the same specific analysed sample volume are recorded in a time-overlapping manner.
  • fluorescence image it is therefore meant that an image of the fluorescent emitted light (i.e. the fluorescent response), that is emitted under the effect of excitation is recorded in the image; where the fluorescence image shows information (to the greatest possible extent) exclusively of those objects that fluoresce (which present a fluorescent response to the fluorescent excitation).
  • the holographic image comprises the hologram of the specific analysed sample volume, and is recorded from the specific analysed sample volume applying a suitable holographic arrangement.
  • a suitable holographic arrangement can be, by way of example, the in-line holographic arrangement illustrated in the embodiment shown in Fig. 1 , but of course other types of holographic arrangements can also be applied.
  • the holographic image is preferably a digital holographic image, and the applied detector is preferably a digital holographic detector. In the case of applying a digital holographic arrangement the benefits of digital holography can be exploited for the purposes of the invention.
  • the fluorescence image and the holographic image are both recorded of the specific analysed sample volume, with the excitation light also being directed to this sample volume.
  • the entire specific analysed sample volume is not necessarily illuminated uniformly by the excitation light, but the excitation light is preferably applied in such a manner that a volume slightly larger than the specific analysed sample volume is illuminated. In this latter case, preferably, all of the fluorescent objects located inside the specific analysed sample volume can be excited.
  • the fluorescence image is therefore recorded from the specific analysed sample volume.
  • recording an image of the specific analysed sample volume it is meant that the fluorescence image will show to the greatest possible extent the specific analysed sample volume (i.e. a two-dimensional projection thereof to the plane of the detector), and that in addition to that the image may show information of regions that lay outside the specific analysed sample volume.
  • the holographic image is meant to be formed in a similar manner (since typically the holographic beam also extends further in a lateral direction than the specific analysed sample volume). It is possible to find the common part of the recorded images in this manner, which part will essentially be the projection (imaging) of the specific analysed sample volume.
  • a fluorescent object identification step is carried out on the fluorescence image, and, in case at least one fluorescent object has been identified in the specific analysed sample volume in the fluorescent object identification step, at least one respective object position is assigned to the at least one fluorescent object on the fluorescence image, and a reconstructed holographic image is generated by reconstructing the holographic image, and a reconstructed model (reconstructed image, three dimensional model) of the at least one fluorescent object is identified on the reconstructed holographic image based on the at least one object position.
  • identifying the reconstructed image we mean that it is determined which separable reconstructed model (from the reconstruction of the complete holographic image) can be assigned to the fluorescent object.
  • the fluorescent object identification step is described in detail below.
  • the fluorescence image is examined in order to establish based on the information recorded therein if there is a detail (even a single pixel) in the image that corresponds to a fluorescent object, i.e. the light emitted by the fluorescent object is imaged on that region of the fluorescence image.
  • the subsequent steps of the method are thus carried out only in case at least one fluorescent object has been identified based on the fluorescence image in the fluorescent object identification step. If there can be found a fluorescent object in the currently present specific analysed sample volume, the imaged pattern, i.e. the image of the light emitted by it (hereinafter: object image) appears in the fluorescence image.
  • This object image can be applied for identifying the fluorescent objects in the fluorescence image. Therefore, in the fluorescent object identification step it has to be established whether the fluorescence image comprises an object image of a fluorescent object, and, if yes, all of the object images have to be identified. Through identification a respective object position can be assigned to each fluorescent object based on the location of the object images. If the currently analysed sample volume does not comprise any fluorescent objects of which there would be a (meaningful) object image on the fluorescence detector, these steps of the method are not carried out (in the case of a flow-through sample, analysing is continued with another sample volume). For small-sized or point-like object images the object position can be determined easily, while in the case of larger-sized object images an object position has to be assigned to the object in some way, e.g. using the geometric centre of the object image.
  • the holographic image corresponding to that particular sample volume is reconstructed.
  • This reconstructed holographic image will comprise the reconstructed image of the identified fluorescent object, i.e. the three-dimensional model thereof.
  • the reconstructed holographic image will also comprise in many cases the reconstructed images (3D models) of further objects that have not been identified as fluorescent objects (holographic reconstruction cannot be carried out selectively only for identified objects).
  • the at least one fluorescent object can advantageously be easily found among the (typically many) reconstructed objects.
  • the apparatus according to the invention searches for fluorescent emitting objects in a sample which is preferably flowing-through type. In the event of detecting such an object in the fluorescence image, the image of the fluorescent object is reconstructed based on the holographic image recorded in parallel with the fluorescence image.
  • the two images are preferably spatially and temporally registered, which means that a region (window) is identified which appears in both images, with the mapping between them (i.e. which position in the second image corresponds to a specific position in the first one) being also determined.
  • the position of the fluorescent (fluorescence- emitting) object determined based on the fluorescence image shows the position (i.e. gives the x, y coordinates) of the object inside the cell, while the exact depth (z coordinate) of the fluorescent object is determined by the holographic reconstruction (however, for finding the given object in the holographic image it is sufficient to know the x and y coordinates).
  • the invention also relates to an apparatus for fluorescence analysis of a sample; the method according to the invention can for example carried out on this apparatus. An embodiment of the apparatus according to the invention is illustrated in Fig. 1.
  • the apparatus shown in the figure comprises a sample holder spatial region 10 adapted for receiving a specific analysed sample volume of the sample being analysed, one or more excitation light sources 12a, 12b adapted for illuminating the sample holder spatial region 10 with fluorescent excitation light, a fluorescence detector 14 adapted for recording a fluorescence image by imaging the fluorescent emitted light of a specific analysed sample volume emitted under the effect of the fluorescent excitation light, the specific analysed sample volume being in the sample holder spatial region 10 during the analysis, and a holographic arrangement adapted for recording holographic images of the specific analysed sample volume, the holographic arrangement comprising a holographic light source 16 and a holographic detector 18.
  • the wall of the sample holder spatial region 10 is typically transparent, i.e. the excitation light, the emitted light and the light of the holographic light source can penetrate through it (preferably without refraction and intensity loss).
  • fluorescence detector refers to a detector adapted for detecting fluorescent emitted light emitted by fluorescent objects, i.e. a detector onto which light originating from a fluorescent source is imaged.
  • holographic detector is used to refer to a detector adapted for recording holographic information. Both detectors (imaging devices) are photodetectors (typically having different resolution and sensitivity) but are applied for different purposes, i.e. they are applied for detecting light from different sources (the emitted fluorescent light and the light of the holographic light source).
  • the adjectives "fluorescence” and “holographic” are therefore applied not for describing an essential feature of the devices but solely for differentiating the two detectors; thus these adjectives can optionally be omitted.
  • fluorescence image and "holographic image”, where the adjectives are included in order to specify the type of light to be imaged in the particular image; thereby the adjectives are included in these terms for the purpose of differentiation, and can optionally be omitted.
  • the above also applies to the fluorescence-imaging lens 22 described below.
  • the fluorescence detector 14 and the holographic detector 18 are arranged on a common optical axis with the sample holder spatial region 10 at the opposite sides of the sample holder spatial region 10.
  • fluorescence and holographic images can be recorded of the sample holder spatial region in a very preferable manner because both detectors have the same view of the sample holder spatial region 10 but from the opposite directions.
  • the apparatus comprises a coupling-in element (directing-in element) 20 adapted for imaging an object wave originating from the holographic light source 16 onto the common optical axis and directing it towards the sample holder spatial region 10, and also adapted for reflecting the light of the holographic light source 16 and transmitting the fluorescent emitted light. Therefore, in a manner illustrated also in Fig. 1 , the fluorescence detector 14 and the holographic detector 18 both face the sample holder spatial region 10, i.e.
  • the apparatus records the fluorescent characteristics of the sample volume located in the sample holder spatial region 10 (i.e., by way of example, of the sample flowing through a measurement cell) from one direction, while from the other direction a - preferably digital - holographic measurement is carried out by the apparatus.
  • a relatively thick (even as thick as 0.8 mm, while maintaining a resolution of 1 ⁇ on the holographic detector) flow-through measurement cell (which comprises the sample holder spatial region under investigation) is preferably applied.
  • the sample holder spatial region for analysation may also reach a thickness of 0.8 mm, which corresponds to its dimension measured along the common optical axis.
  • an optical projection - by way of example, a low amount of optical magnification or demagnification - is preferably applied.
  • Fig. 1 Such an embodiment is illustrated in Fig. 1.
  • the apparatus comprises a fluorescence-imaging lens 22 arranged on the common optical axis between the fluorescence detector 14 and the coupling-in element 20.
  • the magnification/demagnification of the fluorescence-imaging lens 22 is preferably between 0.5 and 1.5, more preferably between 0.9 and 1 .1 , in particular approximately 1.
  • a fluorescence-imaging lens 22 with a magnification/demagnification of approx. 1 is applied, i.e. the lens neither magnifies nor demagnifies. If the depth of focus provided by the lens having a magnification/demagnification of approx.
  • the depth of focus can be increased by applying a imaging lens having a lower magnification/demagnification, potentially even providing a depth of focus nearly equalling the z-direction dimension of the sample holder spatial region. If the application of a lens with a magnification/demagnification around 1 results in a depth of focus larger than the z-direction dimension, the magnification/demagnification value can be increased above 1 , i.e. a magnifying lens can be applied as fluorescence-imaging lens. It is however not necessary to increase the magnification/demagnification because in this case the depth of focus resulting from a magnification/demagnification around 1 is sufficient.
  • a depth of focus of the fluorescence-imaging lens is adjusted, by adjusting the demagnification/magnification value thereof, to a dimension of the sample holder spatial region along the common optical axis (hereinafter: z direction), thanks to which such a depth of focus can be preferably chosen that allows for the essentially complete imaging of the entire depth of the sample holder spatial region onto the fluorescence detector. Therefore, if essentially the entire depth of the sample holder is inside the depth of focus of the imaging then the objects to be analysed remain detectable (will not disappear due to not being focussed) anywhere inside the measurement cell.
  • the entire depth of the sample holder can be imaged onto the fluorescence detector (light beams emitted anywhere in the depth of the sample holder are imaged onto the detector).
  • a further advantage of arranging a fluorescence- imaging lens 22 is that the lens allows for eliminating the intensity reduction (proportionate to the square of the distance from the source) of the emitted fluorescent light, because the fluorescence-imaging lens 22 is applied for imaging onto the fluorescence detector the fluorescent light that has still sufficient intensity.
  • a fluorescence image recorded at low magnification is not suitable for the recognition of the fluorescent objects (due to low magnification and low resolution [and possibly due to being out of focus] the objects will be blurred in the image, the applied magnification not being sufficient for shape recognition due to the small size of the commonly occurring fluorescent objects), it can be applied for detecting and identifying such objects, preferably in large depth of the sample holder. Essentially, only images recorded at high magnification could be appropriate for shape recognition. However, in that case the depth of focus would be very low. At the same time it also has to be taken into account that the light emitted by fluorescent objects can be detected with high confidence if the fluorescence image is recorded with a sufficiently long exposure time (otherwise the object images of the fluorescent objects would typically be really faint). However, relatively longer exposure times can result in that the fluorescent objects are displaced during the exposure time, which would also reduce the chance of recognizing the objects in the fluorescence image. That is why fluorescent flow cytometers are applied almost exclusively for counting the fluorescent objects.
  • the apparatus according to the invention further comprises an appropriately configured holographic arrangement. In case a magnifying lens is also arranged, essentially a holographic microscope can be obtained.
  • the fluorescing objects can already be classified according to morphological aspects, since by determining the position (determination of the object position) in the fluorescence image, the objects can be identified in the reconstructed holographic image based on their location.
  • the fluorescence of the objects comprised by the sample can be analysed applying a camera (see below) having appropriate sensitivity (in known approaches wherein the fluorescence image and the holographic image are recorded by means of the same detector it is not possible to choose the sensitivity of the detector to match the application, i.e. recording fluorescence or holographic images).
  • the position of the fluorescent objects (or more precisely, the 2D projection of their 3D position) inside the specific analysed sample volume can be determined. Because both the fluorescence image and the holographic image are recorded of the same state - of a specific analysed sample volume - the content of the two images can be associated with each other, that is, information corresponding to the same object can be found (registered) in both images.
  • the exposure time of the fluorescence image is typically much longer than the exposure time of the holographic image. Accordingly, the holographic image is recorded with a much shorter exposure time during the exposure of the fluorescence image; the two images are therefore recorded in a time-overlapping manner. This is facilitated by the detection of the position of the fluorescent object in the fluorescence image, because thereby - provided that the two images are properly registered - the position information can be applied for identifying the given objects also in the holographic image.
  • the detector applied for recording the fluorescence image is preferably a high- sensitivity one (so that the relatively weak fluorescent signal can be detected as properly as possible) but has relatively low resolution.
  • the holographic image is preferably recorded using a high-resolution detector that otherwise has low sensitivity compared to the fluorescence detector (the holographic detector is not required to have high sensitivity because the intensity of the light emitted by holographic light source - e.g. laser or LED - is not so low).
  • the sensitivity of the (colour) camera applied for fluorescence measurement was 5.3 V/Lux-sec, while the sensitivity of the camera utilized in the digital holographic arrangement was 0.724 V/Lux-sec.
  • a fluorescence detector with a sensitivity of 4-7 V/Lux-sec, and/or a holographic detector with a sensitivity of 0.5- 2 V/Lux-sec are applied.
  • both the fluorescence detector and the holographic detector can be chosen preferably such that they have a sensitivity that matches the specific application; particularly preferably the sensitivity of both detectors is selected from the above specified ranges, and thereby both detectors of the apparatus can be best suited for the given application.
  • an exposure time (T F ) of 40-200 ms is applied for recording the fluorescence image
  • an exposure time (T H ) of 0.5-5 ms is applied for forming the holographic image.
  • the fluorescence image will comprise with high probability the representation of the light beams originating from weakly emitting fluorescent objects.
  • the choice from within the range is naturally also affected by the intensity of the excitation light, and the extent to which the intensity of the emitted fluorescent light is reduced until the fluorescence detector is reached.
  • the exposure time T H is chosen from the above specified range, the holographic image will be sufficiently sharp with high probability, and the displacement of the fluorescent objects can be avoided. If both exposure time values are selected from the above specified respective ranges then both of these advantages will be present, i.e. both detectors can be tuned such that they match the specific application.
  • a fluorescence detector with a resolution of 0.5-3 megapixels/25 mm 2 is preferably applied. More preferably, a detector with a resolution of 1 -2 megapixels/25 mm 2 can be applied, and a detector with a resolution of approximately 1.3 megapixels/25 mm 2 is particularly preferably applied.
  • an imaging lens is preferably applied for the imaging, the lens may be a magnifying or demagnifying lens.
  • the magnification/demagnification value is approximately 1 , i.e. essentially no magnification or demagnification occurs. In that case, therefore, an image with a size equalling the dimension of the analysed region measured perpendicularly to the common axis the imaging lens is projected onto the fluorescence detector.
  • the image does not typically fill the whole 5x5 mm area of the detector, and therefore only a subregion of the detector, by way of example, with a size of 256x256 pixels or 320x320 pixels, is utilized during the analysis.
  • the holographic image is preferably recorded with an appropriately small exposure time, selecting the exposure time from the above specified range. Because a high-resolution image is required in order that the reconstruction of holographic image has the appropriate amount of detail, a detector having higher resolution than the one utilized for recording the fluorescence image is applied as the holographic detector. However, this detector has a longer readout time compared to the fluorescence detector.
  • the holographic detector preferably has a resolution of 5-20 megapixels/25 mm 2 ; more preferably a resolution of 10-15 megapixels/25 mm 2 , particularly preferably a resolution of approximately 12 megapixels/25 mm 2 . Detectors having such a resolution can record images that have the amount of detail required for good-quality reconstruction.
  • Detectors with relatively high resolution typically have lower sensitivity, but this sensitivity is totally appropriate for detecting the light of the holographic light source.
  • the relatively longer exposure time of the fluorescence image and the longer readout time of the holographic image can be harmonized (they are in the same order of magnitude); both images can be safely read out before starting the next measurement - i.e. before the volume to be analysed is completely replaced. In many cases the time elapsing until recording the next image is even longer than that.
  • the method according to the invention provides significantly faster processing.
  • the holographic image corresponding to a specific fluorescence image is reconstructed only if (at least one) fluorescent object has been successfully identified in the fluorescence image. In case no fluorescent object could be identified in the fluorescence image, the corresponding holographic image is ignored and is not reconstructed.
  • Sample evaluation is also aided by that the objects being fluorescent on a given fluorescence image can be identified by determining the object positions on the fluorescence image (in most cases, with sufficiently sparse samples there is typically only a single such object in a given image).
  • the method and apparatus according to the invention has the great advantage that it allows for generating a high-resolution image (model) of the fluorescent objects (either auto-fluorescent or with dye- induced fluorescence) comprised in the sample flowing through the apparatus, which facilitates increased-accuracy classification.
  • test objects are applied for registering (that is, for associating the images with each other, for determining the common regions imaged by both cameras, and in case the cameras are oriented differently, i.e. not facing with each other in the apparatus according to the invention) the images recorded by the cameras - i.e. by the fluorescence detector and the holographic detector.
  • Test objects e.g. comprise some highly fluorescent objects which can also be easily identified by the holographic system.
  • Test pearls are e.g. applied as test objects. In the event of the simultaneous detection of an appropriate number (for example, three) of such test pearls the objects can be identified based on their reconstructed arrangement.
  • the amount of rotation, shift and magnification required for registering the images of the two detectors can be determined applying an appropriate algorithm, i.e. registration of the detectors can be carried out.
  • the settings obtained thereby can also be tested preferably by applying further test objects. This is a semi-automatic calibration process (requiring human intervention), but alternatively a fully automated calibration process can also be devised and applied.
  • Calibration can also be performed based on the motion patterns of the individual objects, i.e. by tracking the objects (moving in a quite irregular fashion) in a series of images (also by continuously determining, tracking the object positions in both images).
  • low optical magnification/demagnification is required or an imaging lens without magnification/demagnification has to be applied. If no demagnification/magnification is applied, the fluorescence detector has to be arranged in the immediate vicinity of the measurement cell (sample holder).
  • the measurement is made more difficult by the square-law reduction of light intensity with distance resulting in a spread-out patch of light (it would be more and more difficult to detect and locate with increasing distance).
  • an imaging lens can be applied in the invention, which allows that the sample holder spatial region and the fluorescence detector can be arranged further away from each other.
  • Some known holographic systems especially systems wherein the fluorescence image and the holographic image is recorded with the same detector, apply a lens- less arrangement, because there cannot be arranged such a lens which could be suitably applied for both the fluorescence detector and the holographic detector.
  • a sufficiently intense illumination can easily be provided over the entire (smaller) volume.
  • the apparatus comprises a holographic imaging lens 24 arranged between the sample holder spatial region 10 and the holographic detector 18.
  • the magnification of the holographic imaging lens 24 is preferably between 1.5 and 10, more preferably between 2 and 3, and particularly it is approximately 2.5. Applying such magnification values a sufficiently high resolution that is required for good-quality reconstruction can be provided.
  • the fluorescence camera (detector) - where low magnification has to be applied in order to achieve high depth of focus - preferably has low resolution (large pixel size), but has very high sensitivity and is preferably capable of high-gain operation with low noise.
  • the camera is also capable of detecting colours, and can thereby differentiate between fluorescent signals of different colours.
  • holographic camera - as appropriate illumination can be provided more easily due to the typically applied laser or LED light sources - small pixel size and the high achievable resolution is preferred instead of sensitivity.
  • a monochromatic detector can be applied also, because colour information can be replaced by fluorescent signals (even multiple, different-colour fluorescent emissions can be detected) and coloured illumination is hindered by the fluorescence measurement anyway.
  • the optical arrangement according to the invention by which a measurement apparatus much more effective compared to the known approaches can be realized.
  • an appropriate magnification may be needed in order to provide high-resolution reconstruction (with a lens-less setup the maximum resolution is usually limited by the pixel size of the detector), for which in the case of the invention it is possible to apply an objective.
  • the potential resolution is significantly limited in a lens-less arrangement.
  • a lens-less system is capable of processing only samples with little or no motion, low resolution and relatively low thickness.
  • the method and apparatus according to the invention can be preferably applied for the simultaneous detection of multiple fluorescent response types.
  • the wavelengths at which the fluorescent objects emit light are sufficiently far from each other, then these can be differentiated applying a simple colour camera (e.g. one implemented using the Bayer pattern). But in the case where the emission peaks are close to each other the application of more than one fluorescence cameras is conceivable, with the optical paths of the cameras being preferably separated by means of dichroic mirrors (by way of example, if both objects emit red light, but one of them at a wavelengths higher by 30 nm).
  • the sample flows through the measurement cell (the sample holder spatial region).
  • the sample is relatively sparse, it comprises objects that either exhibit auto-fluorescence (e.g.
  • fluorescent dye FDA - fluorescein diacetate, which is a dye adapted for differentiating biotic and abiotic material
  • FITC - fluorescein isothiocyanate which is a fluorescent dye, etc.
  • Fluorescence illumination with an appropriate wavelength is provided by one or more excitation light sources 12a, 12b (cf. Fig. 1 ) that are arranged in an angle and are adapted for illuminating the analysed volume uniformly and with sufficient intensity.
  • the excitation light sources 12a, 12b are arranged such that they are capable of illuminating the sample holder spatial region 10 at a direction different from the common optical axis (i.e. in arranged in an oblique manner, and, preferably at a low angle of approximately 30-60° with respect to the common optical axis).
  • the apparatus comprises excitation light sources 12a, 12b adapted for emitting excitation light with wavelengths being different from each other.
  • excitation light sources can of course be implemented as LEDs but lasers or appropriately filtered halogen light sources can also be applied as excitation light sources.
  • the apparatus comprises excitation filters 26a, 26b transmitting the excitation light (for filtering out all other wavelengths to the greatest possible extent) and arranged between the excitation light source 12a, 12b and the sample holder spatial region 10.
  • excitation filters 26a, 26b transmitting the excitation light (for filtering out all other wavelengths to the greatest possible extent) and arranged between the excitation light source 12a, 12b and the sample holder spatial region 10.
  • fluorescent excitation lights are applied in oblique directions but are focused on the measured spatial region, they can be tuned, modulated and filtered individually. Thereby they are able to provide high intensity and do not interfere with each other.
  • the emitted (auto- or dye-induced) fluorescence is recorded by one (or more) high-sensitivity fluorescence detectors 14 (cameras) through a coupling-in element 20 applying appropriate optical components.
  • An emission filter 28, expediently arranged in front of the fluorescence detector and adapted for transmitting the fluorescent emitted light (and for filtering out other wavelengths as much as possible) provides that only the fluorescent signal can reach the camera, i.e. the excitation light and the light of the holographic light source are filtered out.
  • the apparatus comprises an emission filter 28 transmitting the fluorescent emitted light and being arranged on the common optical axis between the fluorescence detector 14 and the coupling-in element 20.
  • the illumination having a coherence length required for the holographic system, is exemplary provided by a laser.
  • the wavelength of the laser should be chosen such that it does not disturb the fluorescence measurement, and that it can be appropriately separated from the fluorescent illumination by means of a suitable filter, i.e. in the embodiment of Fig. 1 a first filter 30 transmitting the light of the holographic light source 16 (and for filtering out other wavelengths to the greatest possible extent) and being arranged between the holographic detector 18 and the sample holder spatial region 10.
  • the filter 30 provides that only the diffracted images of the objects illuminated by the laser is recorded by the camera and that the fluorescent excitation light is completely cut out.
  • the holographic image of the sample is recorded, through a suitable optical arrangement, by a - preferably digital - high-resolution camera.
  • the parameters of the reconstruction can preferably be determined by processing a reduced-resolution image before reconstructing the full-resolution one, more precisely than what is allowed by optimizing processing times in known approaches. Determining the reconstruction distance as accurately as possible is very important. If there is sufficient time for determining the reconstruction distance, the image of the reconstructed objects can be of much higher quality.
  • the method according to the invention allows for that.
  • the support of the object (the region encompassing the object) can also be determined more precisely by applying the method according to the invention, since the processing time available for reconstructing certain really interesting objects can be increased by as much as an order of magnitude.
  • Samples subjected to fluorescence analysing are typically so sparse that, when the sample is fed to the sample holder in a flow- through regime, a fluorescent object will be found in every tenth image or even more rarely.
  • the time available for reconstructing the selected holographic images increases due to this ratio. If a sample is populated by fluorescent objects more densely than this, it can be diluted to allow for more efficient processing and analysis.
  • the method and apparatus according to the invention are e.g.
  • a fluorescent dye is applied for verifying that the purification process has killed all living organisms.
  • a differential fluorescence image is generated by subtracting the average of the fluorescence images (i.e. the average of the current and the earlier image) from the last recorded (i.e. current) fluorescence image, and carrying out the fluorescent object identification step on the differential fluorescence image.
  • the background fluorescence is preferably averaged out by subtracting the average calculated using the earlier fluorescence images, and thus only the deviation from the average is detected.
  • a threshold value (of light intensity measured in the fluorescence image) is preferably set below which the light intensity values detected during the search are not regarded as corresponding to any fluorescent object.
  • the search for fluorescent objects comprises finding the maximum-intensity location in the fluorescence image. If such a value is found and it is larger than the predetermined threshold, then the measured values are examined in its vicinity, as they may also belong to the given object. Thereby, multiple detections of the same object can be ruled out because the regions belonging to a given object can be excluded from the subsequent search for further fluorescent objects.
  • the search is preferably performed in a pixel-by-pixel manner, examining the light intensity recorded at the given pixel.
  • the 2D-position of an object inside the analysed sample holder spatial region is detected by means of the fluorescence camera.
  • the fluorescent objects are found essentially on the basis of the fluorescence image (and accordingly the non-fluorescent objects can be ignored).
  • information provided by the two images can expediently be compared (by way of example, it can be established whether the contents of both images suggest an extended object).
  • the fluorescent objects can be differentiated from background noise. Because non-moving objects are eliminated by averaging the images and subtracting the average from the current image, preferably only the moving fluorescent objects (being moved inside the flow-through cell or having own locomotion) are shown in the differential fluorescence image resulting from the subtraction.
  • the applied threshold value depends on the fluorescent signal (if the signal is weak, the threshold is low). Accordingly, the threshold value should be adjusted to match the currently applied illumination. It has to be noted that in the case of FDA staining (when the objects are made artificially fluorescent by applying a dye), where background fluorescence is increasing during the measurement, this variation should be taken into account when the threshold value is chosen, i.e. a time-variable threshold value may be needed.
  • a threshold value being variable depending on the background fluorescence of the specific sample volume is applied.
  • the threshold value is dependent on the illumination, on the sensitivity and dynamic range of the camera, the gain of the apparatus, the signal- to-noise ratio and signal strength. Any one of these parameters may change by way of example in case a component is replaced.
  • Figs. 2A-2C The timing of recording the images is illustrated in Figs. 2A-2C also referred to above.
  • the fluorescence images and the holographic images are recorded in a time-overlapping manner. This is illustrated in Figs. 2A and 2B, showing the exposure time T F , and the exposure time T H corresponding respectively to the fluorescence image and the holographic image.
  • Figs. 2A and 2B are intended to be interpreted together. They show the relationship of the typical length of the exposure times TF and T H and also illustrate that the exposure of the two images is started roughly at the same time, thus preferably providing for their overlap in time.
  • T F the holographic image can be recorded with different timing, too.
  • a much longer exposure time is typically applied for the fluorescence image, but because the required camera resolution is lower the combined exposure and readout times are typically the same for the two images (for the fluorescence image the exposure time is long but readout is fast, and conversely, for the holographic image the exposure time is short but image readout takes longer due to the larger resolution). Because the objects can get displaced during the long exposure period, a slight motion blur may occur. This, however, can be neglected due to the low resolution, i.e. it essentially does not affect the ability to identify the specific object in the holographic image.
  • the occurrence of motion blur can also be exploited in a preferred embodiment (illustrated in Fig. 2C).
  • the specific analysed sample volume is illuminated with two fluorescent excitation lights of different wavelengths, and, during another time portion T F i of the exposure time T F , the specific analysed sample volume is illuminated with only one of the two fluorescent excitation lights, the fluorescence image is recorded by imaging the fluorescent emitted light emitted under the effect of the fluorescent excitation lights, and in case at least one fluorescent object has been identified in the fluorescent object identification step, a fluorescent object excited by the fluorescent excitation lights applied during both of the time portions is identified based on the displacement of the at least one fluorescent object on the fluorescence image, and/or a fluorescent object excited by the fluorescent excitation light applied during either one of the time portions is identified.
  • Such an illumination scheme is illustrated in Fig. 2C, showing also the full exposure time T F ; in the case depicted in the figure only one excitation illumination is applied in the time portion T F i (operating for example only the excitation light source 12a), with the other excitation being turned on for the time portion Tf 2 ' , thereby the currently analysed sample volume is illuminated with two excitation lights with different wavelengths. It is also conceivable that two illuminations are applied simultaneously during a first time portion which is followed by applying only one illumination in a second time portion, when one of them is switched off. The steps of the method according to this embodiment can also be carried out on an image recorded in the above manner.
  • This embodiment can therefore be applied for differentiating between fluorescent signals (for example, the red fluorescence of chlorophyll at 680 nm induced by blue illumination at 470 nm from the red fluorescence of phycocyanin at 650 nm induced by amber illumination at 595 nm) that otherwise cannot be differentiated b means of the fluorescence detector (camera) without a special dedicated filter.
  • fluorescent signals for example, the red fluorescence of chlorophyll at 680 nm induced by blue illumination at 470 nm from the red fluorescence of phycocyanin at 650 nm induced by amber illumination at 595 nm
  • the cells that also comprise phycocyanin, which have a special motion blur pattern (the motion blur corresponding to the two objects will be different because the motion recorded in the image are of different duration, and thus the motion blur representing the displacement during the exposure period is also different) can be differentiated by turning on the amber illumination at an appropriate time.
  • FIG. 3A-3B exemplary corresponding fluorescence and holographic images are illustrated, while in Fig. 3C the reconstruction of the holographic image of Fig. 3B is shown.
  • Fig. 3A three objects appear, i.e. during the corresponding analysis three fluorescent objects were situated in the analysed sample holder spatial region at the time the image was recorded.
  • green and red fluorescence can be discerned based on the fluorescence image (the red fluorescence originating in the example from auto-fluorescence, while the green one originating from dye-induced fluorescence), the image of the leftmost object being red and the images of the other two objects being green.
  • the respective object images are indicated by arrows.
  • FIG. 3B shows the holographic image corresponding to the fluorescence image shown in Fig. 3A.
  • Fig. 3B shows the holographic image corresponding to the fluorescence image shown in Fig. 3A.
  • several other object images i.e. the interference patterns corresponding to them.
  • a great advantage of the invention is that by applying the method according to the invention the images corresponding to the fluorescent objects can be selected from the many object images that can be seen in the holographic image of Fig. 3B.
  • the object images corresponding to the fluorescent objects can selected from the object images comprised in the holographic image.
  • Fig. 3C shows the reconstruction of the object image (interference pattern) enclosed in the white box in Fig. 3B, in the figure the reconstructed shape of the fluorescent object can be observed.
  • ring-shaped artefacts can be seen because of the twin-image formed due to the inline arrangement applied for recording the holographic image.
  • the disturbance caused by the twin-image can be reduced applying known approaches, the morphological analysis, i.e. the examination of the three-dimensional model obtained for the fluorescent object is not affected by the rings (the twin-image elimination process is slow, so it is not applied).
  • Fig. 4 shows a fluorescence image recorded with a relatively long exposure time. Due to the displacement of the fluorescent object the object image "leaves a trail", i.e. it exhibits motion blur. According to the above, through modifying the fluorescent excitation in an appropriately timed manner this may allow for differentiating objects that fluoresce at the same wavelength.
  • flow-through samples preferably have very low flow speeds during the exposures (the small-scale motion blur that is discussed above and can be preferably exploited is typically present even with stationary samples).
  • the flow speed of the fluid can be increased applying a pump, followed by reducing the flow speed again for the next exposure. Images are therefore preferably exposed after the sample (set in motion by a pump) has slowed down or halted. The time required for that is not long relative to the applied exposure and readout times, so it does not significantly slow down the operation of the apparatus, i.e. images can preferably be recorded in relatively rapid succession.
  • an exposure is taken every second, but a rate of 3-4 Hz may probably be achieved (in that case it can also be advantageously exploited that the exposure time of the fluorescence image and the readout time of the holographic image are roughly the same, and thus the two processes can be coordinated).
  • the holographic system in the case of a lens- less setup
  • the dichroic mirror essentially an interference filter capable of providing high optical density/suppression
  • object-to-detector distances smaller than this distance is not possible in such known approaches.
  • flow speed has to be adjusted such that any given object is recorded in only one hologram, but it should not exhibit motion blur in it, i.e. it should get displaced by an amount smaller than the pixel size during the exposure period.
  • flow speed has to be adjusted such that any given object is recorded in only one hologram, but it should not exhibit motion blur in it, i.e. it should get displaced by an amount smaller than the pixel size during the exposure period.
  • each object is imaged only once it has to be ensured that the entire analysed volume is replaced between two subsequent exposures.
  • a flow-through moving sample is applied and, after a specific sample volume has been replaced by another one, the steps of the method are carried out on at least one another specific sample volume.
  • the exposure time of the fluorescence detector is e.g. 80 ms (the time practically required for gathering enough light for accurate exposure, but this depends on the camera and the fluorescent illumination), it is desirable that the object is displaced at most by a few pixels during this period of time.
  • the flow speed of the sample has to be reduced to that value. Since the holographic images are recorded applying short exposure times (on the order of 1 ms) such speeds will cause only negligible motion blur. After the image is recorded, the sample can be moved further, even at high speeds as that does not affect the analysed volume.
  • the holographic detector has an exemplary resolution of 10-15 megapixels (of which a region of 2048x2048 pixels is utilized as mentioned above); with the average detector size being 5x5 mm. Images of such size can be typically downloaded to a computer approximately at a rate of 4-5 frame/sec; the download time of a single image being approximately 200 ms, but it may be as high as 500 ms. Although cameras much faster than that do exist, these are very expensive and the processing of the obtained images and holograms is much slower, i.e. it is not worth applying such cameras.
  • the holographic exposure is carried out for example by flashing lasers, with a duration in the order of milliseconds.
  • the fluorescence detector has low resolution (for example, 1-2 megapixels), and due to the differences in magnification only a small portion of it is utilized (e.g. 300x300 pixels), with the detector size being typically 5x5 mm. According to the above, relatively long exposure times and high gain values are to be applied in order that a sufficient amount of light can be gathered from the weak fluorescent signal. Of course in this case the applicable exposure time is also dependent on the intensity of the fluorescent excitation light (with shorter exposure times corresponding to strong illumination).
  • a fluorescence image of 256x256 pixels and a holographic image of 2048x2048 pixels are recorded.
  • the size of the utilized detector region also depends on the magnification/demagnification value of the projections. The difference between detector resolutions may be even greater than this, but that may negatively affect the estimation of object locations in the fluorescence image. With sparse samples, where the objects are separated by larger distances and thus are not overlapping irrespective of their fluorescence, even still greater resolution differences can be applied. Applying appropriate filters it is also possible to simultaneously record multiple different fluorescent signals. For that it has to be provided that the applied illuminations (excitation lights) do not interfere with one another, and that the emitted wavelengths can somehow be separated from one another.
  • two respective illuminations at 470 nm and at 600 nm may be applied (with 10 nm band-pass filters being applied as excitation filters).
  • the applied emission filter transmits light above 500 nm (thereby the excitation light with a wavelength of 470 nm is prevented from reaching the fluorescence detector), and a band-stop filter (a so called 'notch filter') with a bandwidth of 20 nm is applied for filtering out the 600- nm illumination such that also that cannot reach the fluorescence detector.
  • fluorescent illuminations are preferably (i.e. also in the embodiment according to Fig. 1 ) provided obliquely, at low angles. This arrangement allows for the simultaneous application of multiple fluorescent lights, and thus the requirements for the applied dichroic filters (in the embodiment shown in Fig.
  • the coupling-in element 20 may function as a dichroic filter for fluorescent emitted lights, because it preferably transmits to the fluorescence detector only these lights) can be relaxed.
  • the suppression provided by the dichroic filter should be much greater than in the case where the sample is illuminated obliquely, since in the latter case only scattered light has to be filtered out.
  • each of the fluorescent illuminations are implemented as separate modules (the excitation light sources 12a, 12b), their appropriate focusing and bandwidth filtering can be provided, i.e. the emission filter (which in the embodiment of Fig. 1 is the emission filter 28) can be also optimized more easily.
  • An afocal optical arrangement can be applied preferably for recording the holograms.
  • the great advantage of such arrangements, compared to others, is that they provide, constant magnification for different objects inside the system, (i.e. the magnification is not dependent on object position), even if the illumination does not provide plane-wave reference. Thereby magnification calibration is not necessary.
  • the essential feature of the applied arrangement is that instead of applying a lens (objective) for achieving the appropriate magnification, a lens system essentially arranged as a telescope is applied. This is preferable because, although with holographic reconstruction the single-lens system generates a magnification independent of object distance, it does so only if the illumination is parallel.
  • magnification of reconstructed objects will vary with distance (it can be calibrated, but not in a simple manner). However, with an afocal arrangement the magnification will be practically independent of the illumination parameters. This typically does not affect object segmentation, because in our case that is performed applying the optimal object reconstructions.
  • the parameters of optimal reconstruction are established by comparing a number of reconstructions carried out at different distances (of course, calculating only lower- resolution images), applying special focus measures. During carrying out the method according to the invention preferably there is also time for determining the distance corresponding to the given reconstruction in a more accurate manner.
  • a dichroic mirror 31 can also be preferably arranged in the apparatus according to the invention.
  • the dichroic mirror is adapted for diverting the fluorescent signal reflected from the holographic imaging lens (microscope objective).
  • the dichroic mirror 31 is not arranged perpendicular to the common optical axis but slightly inclined with respect to it. Otherwise this signal would also enter the fluorescence camera and would generate an extra image recorded from the other side (but more blurry and defocused) of the measured volume.
  • the filter 30 and the dichroic mirror 31 can preferably be replaced by a single optical element arranged at the position of the dichroic mirror 31 . In this case the filter 30 can be omitted.
  • an in-line arrangement i.e. the reference beam, too, is provided by the illumination.
  • Such an arrangement is preferably applied in case the analysed sample is sparse, i.e. scattering on the measured objects can be treated as small perturbations of the reference beam.
  • the advantages of this arrangement are simplicity and robustness (insensitivity to vibrations).
  • Other arrangements may also be considered (namely, on-axis in-line and off-axis arrangements) but in such arrangements it has to be provided that the reference beam can be directed to the holographic detector along a separate path.
  • the coupling-in element is for example implemented by a dichroic low-pass mirror adapted for reflecting short-wavelength light and transmitting longer wavelengths.
  • a dichroic low-pass mirror adapted for reflecting short-wavelength light and transmitting longer wavelengths.
  • it is adapted for providing coherent illumination for the holographic arrangement, and for directing it first to the sample and then onto the holographic detector (the light of the holographic illumination being reflected by the dichroic low-pass mirror), while it transmits the fluorescent emitted light, which results in that the fluorescence camera imaging can be performed through it.
  • the dichroic low-pass mirror is preferably arranged in a manner also shown in Fig. 1 , at an angle of 45° with respect to the common optical axis.
  • the method and apparatus according to the invention is capable of combining the fluorescence measurement capability of flow cytometers with the advantages provided by holographic (preferably digital holographic) images.
  • digital holographic microscopy it is capable of reconstructing high- resolution images (applicable also for object classification) of the identified objects by selecting - based on the fluorescence image - the fluorescent objects from among the few dozen objects comprised in the image recorded by the holographic microscope.
  • the invention is, of course, not limited to the preferred embodiments described in details above, but further variants, modifications and developments are possible within the scope of protection determined by the claims.

Abstract

The invention is a method for fluorescence analysis of a sample in the course of which a fluorescence image is recorded and a fluorescent object identification step is carried out on the fluorescence image. The invention is, furthermore, is an apparatus for fluorescence analysis of a sample, the apparatus comprising a sample holder spatial region (10), an excitation light sources (12a, 12b), a fluorescence detector (14) adapted for recording a fluorescence image by imaging the fluorescent emitted light emitted under the effect of the fluorescent excitation light of a specific analysed sample volume present in the sample holder spatial region (10) during an analysis, and a holographic arrangement adapted for recording holographic images of the specific analysed sample volume, wherein the holographic arrangement comprising a holographic light source (16) and a holographic detector (18). In the apparatus the fluorescence detector (14) and the holographic detector (18) are arranged on a common optical axis with the sample holder spatial region (10) at opposite sides of the sample holder spatial region (10), and the apparatus comprises a coupling-in element (20) adapted for imaging an object wave originating from the holographic light source (16) onto the common optical axis and directing it towards the sample holder spatial region (10), and also adapted for reflecting the light of the holographic light source (16) and transmitting the fluorescent emitted light.

Description

METHOD AND APPARATUS FOR FLUORESCENCE ANALYSIS OF A SAMPLE
TECHNICAL FIELD
The invention relates to a method and an apparatus for fluorescence analysis of a sample. The sample analysed according to the invention is preferably translucent and is in a liquid state, and the invention is adapted for identifying fluorescent objects in the sample.
BACKGROUND ART
Conventional flow cytometers are capable of high-speed counting of objects having fluorescent characteristics (either auto-fluorescent such as chlorophyll, or marked with a fluorescent dye) in a flowing liquid sample. However, these are typically not capable of forming high-resolution images of the analysed objects.
In addition to establishing the count of the flowing cells or particles the detection of their shape or three-dimensional morphology can also be of significance; this is achieved by analysing the liquid samples comprising the fluorescent object also by holographic means.
There are many known approaches wherein holographic images and fluorescence images (i.e. images showing the fluorescent characteristics) of the liquid sample are recorded one after the other, utilizing a single detector. Such approaches are disclosed in US 2004/0156098 A1 , US 7,362,449 B2 and US 2012/0148140 A1. These approaches have a serious disadvantage, namely that the analysed objects (even due to their natural motion in a stationary liquid) can get displaced between the time instances when the two - fluorescence and holographic - images are recorded. Being sensitive to motion these approaches are not suitable for analysing flowing media. Another great disadvantage of such approaches is that because the holographic and fluorescence images are recorded by means of the same detector, the detector cannot be selected either for fluorescence imaging or for holographic imaging, i.e. the detector parameters cannot be optimized for either image type.
Both a fluorescence image and a holographic image are recorded according to US 2015/0056607 A1 , but because the images are applied for analysing samples located in spatially different regions, the two measurements can only be combined statistically.
In US 2014/0376816 A1 a device adapted for analysing and sorting objects being in a flowing medium is disclosed. In the document an approach is disclosed wherein holographic and fluorescent imaging of a specific spatial region is performed without applying an imaging optics, the images are recorded at adjacent regions of the same detector. In order that the images corresponding to the holographic detector part and to the fluorescence detector part can be projected to different regions of the detector, the detector has to be situated relatively further away from the cell comprising the sample to be analysed. However, in the lack of an imaging optics, recording of only an extremely weak fluorescent signal is possible, and the resolution of the holographic image will be also very poor. In addition to that, the detector itself and its operating parameters cannot be specifically optimized for either one of the fluorescence or the holographic measurements since it has to perform both functions at the same time.
In US 2014/0376816 A1 a configuration is disclosed wherein an imaging zone is divided into two parts along a fluidic channel. One of these two parts is adapted for holographic imaging, while the other part is adapted for recording fluorescence images. The images are therefore recorded along a fluidic channel. In both of the above described configurations disclosed in US 2014/0376816 A1 the holographic image and the fluorescence image are recorded in different regions of the same detector.
In certain known approaches a single camera is applied for recording both fluorescence and holographic image; however, the intensity of the light to be detected (which determines the necessary exposure time and gain/sensitivity for detection) is very different in these cases, so the resolution required for recording the two images would need to be different in an optimal case, but that is not feasible due to the common camera detector. Moreover, in these known approaches the simultaneous detection of multiple different types of fluorescent signal is also very troublesome (or is not feasible at all). Due to the common detector synchronous measurements cannot be performed, and thus it is almost impossible to register the fluorescence and holographic images to each other in case of a flow-through sample (it is only possible, with a large error, with an extremely slow flow).
In light of the known approaches the need has arisen for a method and an apparatus that allow for detailed, high resolution reconstruction of the fluorescent objects comprised in a sample, and also for their separation from non-fluorescent objects.
DESCRIPTION OF THE INVENTION
The primary object of the invention is to provide a method and an apparatus which is free from disadvantages of prior art approaches to the greatest possible extent.
A further object of the invention is to provide a method and an apparatus that are capable of generating holographic images of fluorescent objects with sufficient resolution and of differentiating the fluorescent objects from non-fluorescent ones.
A further object of the invention is to provide a method and an apparatus wherein, in addition to fulfilling the above objects, the resource demand of holographic reconstruction can be significantly reduced compared to known approaches.
An object of the invention is to provide a method and an apparatus wherein the parameters of the applied fluorescence detector and holographic detector can be chosen independently of each other in order to fulfil the requirements for such devices.
The objects according to the invention can be achieved by the method according to claim 1 and the apparatus according to claim 8. Preferred embodiments of the invention are defined in the dependent claims.
BRIEF DESCRIPTION OF THE DRAWINGS
Preferred embodiments of the invention are described below by way of example with reference to the following drawings, where
Fig. 1 is a block diagram illustrating an embodiment of the apparatus according to the invention, Figs. 2A-2C are schematic diagrams illustrating the schedule for recording images in an embodiment of the method according to the invention,
Fig. 3A is an exemplary fluorescence image,
Fig. 3B is a holographic image corresponding to Fig. 3A,
Fig. 3C is the reconstruction of the boxed detail of Fig. 3B, and
Fig. 4 is another exemplary fluorescence image.
MODES FOR CARRYING OUT THE INVENTION
The method according to the invention is adapted for fluorescence analysis of a sample. As it is described below, the method according to the invention can be particularly preferably applied for analysing liquid, transparent samples. In such a sample fluorescent objects are typically carried by water as a liquid, transparent medium. When subjected to suitable excitation, the fluorescent objects emit fluorescent light, i.e. they give a fluorescent response. The fluorescent response has a wavelength that is different from the wavelength of excitation light, the response typically having a longer wavelength than the excitation.
The sample analysed applying the method according to the invention is passed through the sample holder in a flow-through manner, at a suitable (suitably low) speed.
In addition to that, the method according to the invention is of course suitable for analysing stationary samples. The method according to the invention is also suitable for analysing solid samples that are sufficiently thin for the fluorescent objects comprised therein to be excited and that the light emitted by them can be detected. A holographic image that is required for carrying out the method can be typically recorded (taken) from such thin samples. Applying sufficiently short exposure times the method and apparatus according to the invention are also suitable for analysing gaseous samples.
The method according to the invention may preferably be carried out applying the apparatus according to the invention, an embodiment of which is illustrated in Fig. 1. ln the course of the method according to the invention a specific analysed sample volume of the sample (i.e. the sample volume currently analysed) is illuminated with fluorescent excitation light, and a fluorescence image is recorded by imaging (projecting) the fluorescent emitted light of the specific analysed sample volume emitted under the effect of the fluorescent excitation light. The fluorescent emitted light can be imaged directly (without an imaging element) onto the plane of the detector, or - as it will be shown later on - preferably applying an imaging lens. In the embodiment illustrated in Fig. 1 the specific analysed sample volume is that portion of the sample which is located inside a sample holder spatial region 10 during the excitation.
The term "fluorescence image" is used herein to refer to an image recorded by imaging fluorescent emitted light emitted by fluorescent objects onto a detector, i.e. the image is adapted for displaying the imaging of light originating from a fluorescent source. The term "holographic image" is used to refer to an image carrying holographic information; i.e. the holographic image comprises information that can be reconstructed from the recorded hologram.
In the course of the method according to the invention, a holographic image of the specific analysed sample volume is recorded overlapping in time with recording the fluorescence image. In the method according to the invention, both a holographic image and a fluorescence image of the same specific analysed sample volume are recorded in a time-overlapping manner.
By fluorescence image it is therefore meant that an image of the fluorescent emitted light (i.e. the fluorescent response), that is emitted under the effect of excitation is recorded in the image; where the fluorescence image shows information (to the greatest possible extent) exclusively of those objects that fluoresce (which present a fluorescent response to the fluorescent excitation).
Further, the holographic image comprises the hologram of the specific analysed sample volume, and is recorded from the specific analysed sample volume applying a suitable holographic arrangement. Such an arrangement can be, by way of example, the in-line holographic arrangement illustrated in the embodiment shown in Fig. 1 , but of course other types of holographic arrangements can also be applied. The holographic image is preferably a digital holographic image, and the applied detector is preferably a digital holographic detector. In the case of applying a digital holographic arrangement the benefits of digital holography can be exploited for the purposes of the invention. The fluorescence image and the holographic image are both recorded of the specific analysed sample volume, with the excitation light also being directed to this sample volume.
By its nature the entire specific analysed sample volume is not necessarily illuminated uniformly by the excitation light, but the excitation light is preferably applied in such a manner that a volume slightly larger than the specific analysed sample volume is illuminated. In this latter case, preferably, all of the fluorescent objects located inside the specific analysed sample volume can be excited.
The fluorescence image is therefore recorded from the specific analysed sample volume. By "recording an image of the specific analysed sample volume" it is meant that the fluorescence image will show to the greatest possible extent the specific analysed sample volume (i.e. a two-dimensional projection thereof to the plane of the detector), and that in addition to that the image may show information of regions that lay outside the specific analysed sample volume. The holographic image is meant to be formed in a similar manner (since typically the holographic beam also extends further in a lateral direction than the specific analysed sample volume). It is possible to find the common part of the recorded images in this manner, which part will essentially be the projection (imaging) of the specific analysed sample volume.
In the course of the method according to the invention, furthermore, a fluorescent object identification step is carried out on the fluorescence image, and, in case at least one fluorescent object has been identified in the specific analysed sample volume in the fluorescent object identification step, at least one respective object position is assigned to the at least one fluorescent object on the fluorescence image, and a reconstructed holographic image is generated by reconstructing the holographic image, and a reconstructed model (reconstructed image, three dimensional model) of the at least one fluorescent object is identified on the reconstructed holographic image based on the at least one object position. By identifying the reconstructed image we mean that it is determined which separable reconstructed model (from the reconstruction of the complete holographic image) can be assigned to the fluorescent object. The fluorescent object identification step is described in detail below. In this step the fluorescence image is examined in order to establish based on the information recorded therein if there is a detail (even a single pixel) in the image that corresponds to a fluorescent object, i.e. the light emitted by the fluorescent object is imaged on that region of the fluorescence image. The subsequent steps of the method are thus carried out only in case at least one fluorescent object has been identified based on the fluorescence image in the fluorescent object identification step. If there can be found a fluorescent object in the currently present specific analysed sample volume, the imaged pattern, i.e. the image of the light emitted by it (hereinafter: object image) appears in the fluorescence image. This object image can be applied for identifying the fluorescent objects in the fluorescence image. Therefore, in the fluorescent object identification step it has to be established whether the fluorescence image comprises an object image of a fluorescent object, and, if yes, all of the object images have to be identified. Through identification a respective object position can be assigned to each fluorescent object based on the location of the object images. If the currently analysed sample volume does not comprise any fluorescent objects of which there would be a (meaningful) object image on the fluorescence detector, these steps of the method are not carried out (in the case of a flow-through sample, analysing is continued with another sample volume). For small-sized or point-like object images the object position can be determined easily, while in the case of larger-sized object images an object position has to be assigned to the object in some way, e.g. using the geometric centre of the object image.
In case, therefore, at least one fluorescent object has been identified (found) in the specific analysed sample volume, the holographic image corresponding to that particular sample volume is reconstructed. This reconstructed holographic image will comprise the reconstructed image of the identified fluorescent object, i.e. the three-dimensional model thereof. However, the reconstructed holographic image will also comprise in many cases the reconstructed images (3D models) of further objects that have not been identified as fluorescent objects (holographic reconstruction cannot be carried out selectively only for identified objects). Applying the method according to the invention the at least one fluorescent object can advantageously be easily found among the (typically many) reconstructed objects. Since both the fluorescence image and the holographic image were recorded of the same specific analysed sample volume, the position in the holographic image corresponding to the identified object position on the fluorescence image can be determined. And, based on the position in the holographic image determined this way the reconstructed image (3D model) corresponding to the fluorescent object can be obtained. Naturally, if there are more than one fluorescent objects present in the image, more than one corresponding 3D models are selected applying the above steps. In operation, the apparatus according to the invention therefore searches for fluorescent emitting objects in a sample which is preferably flowing-through type. In the event of detecting such an object in the fluorescence image, the image of the fluorescent object is reconstructed based on the holographic image recorded in parallel with the fluorescence image. Reconstruction is facilitated by the fact that the two images, recorded in a time-overlapping manner show essentially the same sample volume, and thereby the position identified in the fluorescence image can also be found in the holographic image. To allow for finding the object position, as described above, the two images are preferably spatially and temporally registered, which means that a region (window) is identified which appears in both images, with the mapping between them (i.e. which position in the second image corresponds to a specific position in the first one) being also determined.
To sum up, it can be maintained that the position of the fluorescent (fluorescence- emitting) object determined based on the fluorescence image shows the position (i.e. gives the x, y coordinates) of the object inside the cell, while the exact depth (z coordinate) of the fluorescent object is determined by the holographic reconstruction (however, for finding the given object in the holographic image it is sufficient to know the x and y coordinates). As it was put forward above, the invention also relates to an apparatus for fluorescence analysis of a sample; the method according to the invention can for example carried out on this apparatus. An embodiment of the apparatus according to the invention is illustrated in Fig. 1. The apparatus shown in the figure comprises a sample holder spatial region 10 adapted for receiving a specific analysed sample volume of the sample being analysed, one or more excitation light sources 12a, 12b adapted for illuminating the sample holder spatial region 10 with fluorescent excitation light, a fluorescence detector 14 adapted for recording a fluorescence image by imaging the fluorescent emitted light of a specific analysed sample volume emitted under the effect of the fluorescent excitation light, the specific analysed sample volume being in the sample holder spatial region 10 during the analysis, and a holographic arrangement adapted for recording holographic images of the specific analysed sample volume, the holographic arrangement comprising a holographic light source 16 and a holographic detector 18. The wall of the sample holder spatial region 10 is typically transparent, i.e. the excitation light, the emitted light and the light of the holographic light source can penetrate through it (preferably without refraction and intensity loss).
The term "fluorescence detector" refers to a detector adapted for detecting fluorescent emitted light emitted by fluorescent objects, i.e. a detector onto which light originating from a fluorescent source is imaged. The term "holographic detector" is used to refer to a detector adapted for recording holographic information. Both detectors (imaging devices) are photodetectors (typically having different resolution and sensitivity) but are applied for different purposes, i.e. they are applied for detecting light from different sources (the emitted fluorescent light and the light of the holographic light source). The adjectives "fluorescence" and "holographic" are therefore applied not for describing an essential feature of the devices but solely for differentiating the two detectors; thus these adjectives can optionally be omitted. This also holds for the terms "fluorescence image" and "holographic image", where the adjectives are included in order to specify the type of light to be imaged in the particular image; thereby the adjectives are included in these terms for the purpose of differentiation, and can optionally be omitted. The above also applies to the fluorescence-imaging lens 22 described below.
In this embodiment of the apparatus according to the invention the fluorescence detector 14 and the holographic detector 18 are arranged on a common optical axis with the sample holder spatial region 10 at the opposite sides of the sample holder spatial region 10. With such an arrangement fluorescence and holographic images can be recorded of the sample holder spatial region in a very preferable manner because both detectors have the same view of the sample holder spatial region 10 but from the opposite directions. In the embodiment illustrated in the figure the apparatus according to the invention comprises a coupling-in element (directing-in element) 20 adapted for imaging an object wave originating from the holographic light source 16 onto the common optical axis and directing it towards the sample holder spatial region 10, and also adapted for reflecting the light of the holographic light source 16 and transmitting the fluorescent emitted light. Therefore, in a manner illustrated also in Fig. 1 , the fluorescence detector 14 and the holographic detector 18 both face the sample holder spatial region 10, i.e. applying these detectors the apparatus records the fluorescent characteristics of the sample volume located in the sample holder spatial region 10 (i.e., by way of example, of the sample flowing through a measurement cell) from one direction, while from the other direction a - preferably digital - holographic measurement is carried out by the apparatus.
Since the typical applications of the method and apparatus according to the invention (such as water quality analysing) involve examining large volumes, a relatively thick (even as thick as 0.8 mm, while maintaining a resolution of 1 μηι on the holographic detector) flow-through measurement cell (which comprises the sample holder spatial region under investigation) is preferably applied. Accordingly, the sample holder spatial region for analysation may also reach a thickness of 0.8 mm, which corresponds to its dimension measured along the common optical axis. In order that the fluorescent emitted light can be measured inside such a cell at a large depth applying a digital camera, an optical projection - by way of example, a low amount of optical magnification or demagnification - is preferably applied. Such an embodiment is illustrated in Fig. 1. In this embodiment the apparatus comprises a fluorescence-imaging lens 22 arranged on the common optical axis between the fluorescence detector 14 and the coupling-in element 20. The magnification/demagnification of the fluorescence-imaging lens 22 is preferably between 0.5 and 1.5, more preferably between 0.9 and 1 .1 , in particular approximately 1. Typically a fluorescence-imaging lens 22 with a magnification/demagnification of approx. 1 is applied, i.e. the lens neither magnifies nor demagnifies. If the depth of focus provided by the lens having a magnification/demagnification of approx. 1 would be smaller than the dimension of the sample holder spatial region 10 measured in the direction of the common optical axis, then the depth of focus can be increased by applying a imaging lens having a lower magnification/demagnification, potentially even providing a depth of focus nearly equalling the z-direction dimension of the sample holder spatial region. If the application of a lens with a magnification/demagnification around 1 results in a depth of focus larger than the z-direction dimension, the magnification/demagnification value can be increased above 1 , i.e. a magnifying lens can be applied as fluorescence-imaging lens. It is however not necessary to increase the magnification/demagnification because in this case the depth of focus resulting from a magnification/demagnification around 1 is sufficient.
The above described details mean that in an embodiment of the method and apparatus a depth of focus of the fluorescence-imaging lens is adjusted, by adjusting the demagnification/magnification value thereof, to a dimension of the sample holder spatial region along the common optical axis (hereinafter: z direction), thanks to which such a depth of focus can be preferably chosen that allows for the essentially complete imaging of the entire depth of the sample holder spatial region onto the fluorescence detector. Therefore, if essentially the entire depth of the sample holder is inside the depth of focus of the imaging then the objects to be analysed remain detectable (will not disappear due to not being focussed) anywhere inside the measurement cell.
Thanks to the application of the fluorescence-imaging lens 22 shown in Fig. 1 and to the appropriate choice of parameters (magnification/demagnification, depth of focus) the entire depth of the sample holder can be imaged onto the fluorescence detector (light beams emitted anywhere in the depth of the sample holder are imaged onto the detector). A further advantage of arranging a fluorescence- imaging lens 22 is that the lens allows for eliminating the intensity reduction (proportionate to the square of the distance from the source) of the emitted fluorescent light, because the fluorescence-imaging lens 22 is applied for imaging onto the fluorescence detector the fluorescent light that has still sufficient intensity. In known fluorescence analysing apparatuses wherein the fluorescence image and the holographic image are recorded by the same detector it is usually not feasible to arrange such a fluorescence-imaging lens that can provide the required depth of focus, all the more so because in such systems it is preferred to apply a high- magnification lens for the holographic image.
Although a fluorescence image recorded at low magnification is not suitable for the recognition of the fluorescent objects (due to low magnification and low resolution [and possibly due to being out of focus] the objects will be blurred in the image, the applied magnification not being sufficient for shape recognition due to the small size of the commonly occurring fluorescent objects), it can be applied for detecting and identifying such objects, preferably in large depth of the sample holder. Essentially, only images recorded at high magnification could be appropriate for shape recognition. However, in that case the depth of focus would be very low. At the same time it also has to be taken into account that the light emitted by fluorescent objects can be detected with high confidence if the fluorescence image is recorded with a sufficiently long exposure time (otherwise the object images of the fluorescent objects would typically be really faint). However, relatively longer exposure times can result in that the fluorescent objects are displaced during the exposure time, which would also reduce the chance of recognizing the objects in the fluorescence image. That is why fluorescent flow cytometers are applied almost exclusively for counting the fluorescent objects.
It will be demonstrated below that the above described characteristic features of fluorescence image recording do not present themselves as problems in the method and apparatus according to the invention, since according to the invention the fluorescence image is applied solely for identifying the fluorescent objects (i.e. not for their recognition), for which it is essentially sufficient if the imaging, the object image of the fluorescent objects appears in the fluorescence image. As it will be shown below, according to the invention neither the relatively long exposure time required for recording fluorescence images poses any problem. In order to perform object recognition, morphological classification, and to evaluate the fluorescent content of the sample, the apparatus according to the invention further comprises an appropriately configured holographic arrangement. In case a magnifying lens is also arranged, essentially a holographic microscope can be obtained. Based on the image reconstructed from the holographic image (the spatial model of the analysed objects) the fluorescing objects can already be classified according to morphological aspects, since by determining the position (determination of the object position) in the fluorescence image, the objects can be identified in the reconstructed holographic image based on their location.
In contrast to conventional flow cytometers, in the case of the method and apparatus according to the invention it is not a problem if multiple fluorescent objects are present simultaneously in the analysed sample volume (i.e. the amount of sample located inside the measurement cell/sample holder during the measurement). According to the above, the fluorescence of the objects comprised by the sample can be analysed applying a camera (see below) having appropriate sensitivity (in known approaches wherein the fluorescence image and the holographic image are recorded by means of the same detector it is not possible to choose the sensitivity of the detector to match the application, i.e. recording fluorescence or holographic images).
Based on the image recorded by the camera adapted for recording the fluorescence image, the position of the fluorescent objects (or more precisely, the 2D projection of their 3D position) inside the specific analysed sample volume can be determined. Because both the fluorescence image and the holographic image are recorded of the same state - of a specific analysed sample volume - the content of the two images can be associated with each other, that is, information corresponding to the same object can be found (registered) in both images. The exposure time of the fluorescence image is typically much longer than the exposure time of the holographic image. Accordingly, the holographic image is recorded with a much shorter exposure time during the exposure of the fluorescence image; the two images are therefore recorded in a time-overlapping manner. This is facilitated by the detection of the position of the fluorescent object in the fluorescence image, because thereby - provided that the two images are properly registered - the position information can be applied for identifying the given objects also in the holographic image.
The detector applied for recording the fluorescence image is preferably a high- sensitivity one (so that the relatively weak fluorescent signal can be detected as properly as possible) but has relatively low resolution. The holographic image is preferably recorded using a high-resolution detector that otherwise has low sensitivity compared to the fluorescence detector (the holographic detector is not required to have high sensitivity because the intensity of the light emitted by holographic light source - e.g. laser or LED - is not so low). In an exemplary realization the sensitivity of the (colour) camera applied for fluorescence measurement was 5.3 V/Lux-sec, while the sensitivity of the camera utilized in the digital holographic arrangement was 0.724 V/Lux-sec.
In an embodiment of the apparatus and method a fluorescence detector with a sensitivity of 4-7 V/Lux-sec, and/or a holographic detector with a sensitivity of 0.5- 2 V/Lux-sec are applied. In this embodiment both the fluorescence detector and the holographic detector can be chosen preferably such that they have a sensitivity that matches the specific application; particularly preferably the sensitivity of both detectors is selected from the above specified ranges, and thereby both detectors of the apparatus can be best suited for the given application. In an embodiment of the invention an exposure time (TF) of 40-200 ms is applied for recording the fluorescence image, and/or an exposure time (TH) of 0.5-5 ms is applied for forming the holographic image.
If the exposure time TF is chosen from the above specified range, the fluorescence image will comprise with high probability the representation of the light beams originating from weakly emitting fluorescent objects. The choice from within the range is naturally also affected by the intensity of the excitation light, and the extent to which the intensity of the emitted fluorescent light is reduced until the fluorescence detector is reached. If the exposure time TH is chosen from the above specified range, the holographic image will be sufficiently sharp with high probability, and the displacement of the fluorescent objects can be avoided. If both exposure time values are selected from the above specified respective ranges then both of these advantages will be present, i.e. both detectors can be tuned such that they match the specific application.
In addition to applying a longer exposure time for the fluorescence image it is preferred to apply a lower-resolution detector for recording the fluorescent information, both because a high-resolution result is not needed and because a low-resolution image can be read out from the detector quickly. Besides that, low- resolution detectors typically have high sensitivity due to their relatively large size pixels, and thus all the requirements for the fluorescence detector can be fulfilled by applying a low-resolution detector. Therefore, a fluorescence detector with a resolution of 0.5-3 megapixels/25 mm2 is preferably applied. More preferably, a detector with a resolution of 1 -2 megapixels/25 mm2 can be applied, and a detector with a resolution of approximately 1.3 megapixels/25 mm2 is particularly preferably applied.
The light emitted by the fluorescent objects is imaged onto this detector. As detailed above, an imaging lens is preferably applied for the imaging, the lens may be a magnifying or demagnifying lens. The magnification/demagnification value is approximately 1 , i.e. essentially no magnification or demagnification occurs. In that case, therefore, an image with a size equalling the dimension of the analysed region measured perpendicularly to the common axis the imaging lens is projected onto the fluorescence detector. The image does not typically fill the whole 5x5 mm area of the detector, and therefore only a subregion of the detector, by way of example, with a size of 256x256 pixels or 320x320 pixels, is utilized during the analysis.
The holographic image is preferably recorded with an appropriately small exposure time, selecting the exposure time from the above specified range. Because a high-resolution image is required in order that the reconstruction of holographic image has the appropriate amount of detail, a detector having higher resolution than the one utilized for recording the fluorescence image is applied as the holographic detector. However, this detector has a longer readout time compared to the fluorescence detector. The holographic detector preferably has a resolution of 5-20 megapixels/25 mm2; more preferably a resolution of 10-15 megapixels/25 mm2, particularly preferably a resolution of approximately 12 megapixels/25 mm2. Detectors having such a resolution can record images that have the amount of detail required for good-quality reconstruction. Detectors with relatively high resolution typically have lower sensitivity, but this sensitivity is totally appropriate for detecting the light of the holographic light source. The relatively longer exposure time of the fluorescence image and the longer readout time of the holographic image can be harmonized (they are in the same order of magnitude); both images can be safely read out before starting the next measurement - i.e. before the volume to be analysed is completely replaced. In many cases the time elapsing until recording the next image is even longer than that.
Compared to conventional digital holographic microscopes - where all the objects have to be reconstructed and all the object shapes have to be analysed - the method according to the invention provides significantly faster processing. In contrast to conventional approaches, according to the invention the holographic image corresponding to a specific fluorescence image is reconstructed only if (at least one) fluorescent object has been successfully identified in the fluorescence image. In case no fluorescent object could be identified in the fluorescence image, the corresponding holographic image is ignored and is not reconstructed. Sample evaluation is also aided by that the objects being fluorescent on a given fluorescence image can be identified by determining the object positions on the fluorescence image (in most cases, with sufficiently sparse samples there is typically only a single such object in a given image). With the help of the obtained object positions the reconstructed information corresponding to such objects can be easily isolated in the reconstructed holographic images. Compared to flow cytometers the method and apparatus according to the invention has the great advantage that it allows for generating a high-resolution image (model) of the fluorescent objects (either auto-fluorescent or with dye- induced fluorescence) comprised in the sample flowing through the apparatus, which facilitates increased-accuracy classification.
Preferably, test objects are applied for registering (that is, for associating the images with each other, for determining the common regions imaged by both cameras, and in case the cameras are oriented differently, i.e. not facing with each other in the apparatus according to the invention) the images recorded by the cameras - i.e. by the fluorescence detector and the holographic detector. Test objects e.g. comprise some highly fluorescent objects which can also be easily identified by the holographic system. Test pearls are e.g. applied as test objects. In the event of the simultaneous detection of an appropriate number (for example, three) of such test pearls the objects can be identified based on their reconstructed arrangement. Based on the object positions measured by both the fluorescence detector and the holographic detector (after choosing the appropriate region of interest [ROI]) the amount of rotation, shift and magnification required for registering the images of the two detectors can be determined applying an appropriate algorithm, i.e. registration of the detectors can be carried out. The settings obtained thereby can also be tested preferably by applying further test objects. This is a semi-automatic calibration process (requiring human intervention), but alternatively a fully automated calibration process can also be devised and applied.
Calibration can also be performed based on the motion patterns of the individual objects, i.e. by tracking the objects (moving in a quite irregular fashion) in a series of images (also by continuously determining, tracking the object positions in both images). For the successful volumetric detection of a fluorescent signal (with high depth of focus), low optical magnification/demagnification is required or an imaging lens without magnification/demagnification has to be applied. If no demagnification/magnification is applied, the fluorescence detector has to be arranged in the immediate vicinity of the measurement cell (sample holder). In the case where the detector is arranged further from the objects to be measured the measurement is made more difficult by the square-law reduction of light intensity with distance resulting in a spread-out patch of light (it would be more and more difficult to detect and locate with increasing distance). In such cases an imaging lens can be applied in the invention, which allows that the sample holder spatial region and the fluorescence detector can be arranged further away from each other. Some known holographic systems, especially systems wherein the fluorescence image and the holographic image is recorded with the same detector, apply a lens- less arrangement, because there cannot be arranged such a lens which could be suitably applied for both the fluorescence detector and the holographic detector. Although with a lens-less solution the measurable surface area can be increased, but at the same time it is much more difficult to provide sufficiently uniform, high- intensity fluorescent illumination over a larger surface area. In addition to that, in conventional lens-less solutions a super-resolution imaging technique has to be applied in order to achieve the resolution required for holographic reconstruction because in a lens-less system the primary resolution-limiting factor is the pixel size. Super-resolution imaging requires recording multiple images, which significantly slows down sampling and makes more complex the registration of the corresponding data of fluorescence images.
As it is illustrated by the embodiment shown in Fig. 1 , by utilizing appropriately chosen optical components (objectives, lenses) a sufficiently intense illumination can easily be provided over the entire (smaller) volume.
Due to the separate fluorescence and holographic detectors, according to the invention it is possible to apply an appropriate magnification with which the required resolution can be achieved in a much simpler way also in the holographic system, and thereby the measurement can be performed much more quickly. Accordingly, in the embodiment illustrated in Fig. 1 the apparatus according to the invention comprises a holographic imaging lens 24 arranged between the sample holder spatial region 10 and the holographic detector 18. The magnification of the holographic imaging lens 24 is preferably between 1.5 and 10, more preferably between 2 and 3, and particularly it is approximately 2.5. Applying such magnification values a sufficiently high resolution that is required for good-quality reconstruction can be provided. Although in the case where lenses are applied the measurable surface area and volume may seem smaller, due to the many exposures required with the lens-less arrangement (with super-resolution imaging the number of exposures increases in proportion to the square of resolution) and the low depth of focus of the fluorescence images the larger analysed sample volumes cannot be exploited at all.
As shown above, the requirements for the fluorescence and holographic detectors are completely different. In the arrangement according to the invention the fluorescence camera (detector) - where low magnification has to be applied in order to achieve high depth of focus - preferably has low resolution (large pixel size), but has very high sensitivity and is preferably capable of high-gain operation with low noise. Expediently the camera is also capable of detecting colours, and can thereby differentiate between fluorescent signals of different colours.
In the case of the holographic camera - as appropriate illumination can be provided more easily due to the typically applied laser or LED light sources - small pixel size and the high achievable resolution is preferred instead of sensitivity. Here a monochromatic detector can be applied also, because colour information can be replaced by fluorescent signals (even multiple, different-colour fluorescent emissions can be detected) and coloured illumination is hindered by the fluorescence measurement anyway.
It is readily apparent that the different requirements for the detectors could only be aligned with great compromise.
It is therefore expedient to apply the optical arrangement according to the invention, by which a measurement apparatus much more effective compared to the known approaches can be realized. For the holographic measurement an appropriate magnification may be needed in order to provide high-resolution reconstruction (with a lens-less setup the maximum resolution is usually limited by the pixel size of the detector), for which in the case of the invention it is possible to apply an objective. (It should be noted that the potential resolution is significantly limited in a lens-less arrangement. A lens-less system is capable of processing only samples with little or no motion, low resolution and relatively low thickness.) The method and apparatus according to the invention can be preferably applied for the simultaneous detection of multiple fluorescent response types. If the wavelengths at which the fluorescent objects emit light are sufficiently far from each other, then these can be differentiated applying a simple colour camera (e.g. one implemented using the Bayer pattern). But in the case where the emission peaks are close to each other the application of more than one fluorescence cameras is conceivable, with the optical paths of the cameras being preferably separated by means of dichroic mirrors (by way of example, if both objects emit red light, but one of them at a wavelengths higher by 30 nm). In the method and apparatus according to the invention the sample flows through the measurement cell (the sample holder spatial region). The sample is relatively sparse, it comprises objects that either exhibit auto-fluorescence (e.g. chlorophyll, phycocyanin, phycoerythrin, etc.) or are marked with fluorescent dye (FDA - fluorescein diacetate, which is a dye adapted for differentiating biotic and abiotic material, FITC - fluorescein isothiocyanate, which is a fluorescent dye, etc.).
Fluorescence illumination with an appropriate wavelength is provided by one or more excitation light sources 12a, 12b (cf. Fig. 1 ) that are arranged in an angle and are adapted for illuminating the analysed volume uniformly and with sufficient intensity. In the embodiment according to Fig. 1 the excitation light sources 12a, 12b are arranged such that they are capable of illuminating the sample holder spatial region 10 at a direction different from the common optical axis (i.e. in arranged in an oblique manner, and, preferably at a low angle of approximately 30-60° with respect to the common optical axis). In this embodiment, therefore, the apparatus comprises excitation light sources 12a, 12b adapted for emitting excitation light with wavelengths being different from each other.
The excitation light sources can of course be implemented as LEDs but lasers or appropriately filtered halogen light sources can also be applied as excitation light sources.
In the embodiment illustrated in Fig. 1 the apparatus according to the invention comprises excitation filters 26a, 26b transmitting the excitation light (for filtering out all other wavelengths to the greatest possible extent) and arranged between the excitation light source 12a, 12b and the sample holder spatial region 10. By applying such filters it can be provided that the illumination is optimal for appropriately inducing fluorescence and it does not interfere with holographic and fluorescence measurements, i.e. that only excitation light with the appropriate wavelength can reach the sample holder spatial region 10.
Because fluorescent excitation lights are applied in oblique directions but are focused on the measured spatial region, they can be tuned, modulated and filtered individually. Thereby they are able to provide high intensity and do not interfere with each other. The emitted (auto- or dye-induced) fluorescence is recorded by one (or more) high-sensitivity fluorescence detectors 14 (cameras) through a coupling-in element 20 applying appropriate optical components. An emission filter 28, expediently arranged in front of the fluorescence detector and adapted for transmitting the fluorescent emitted light (and for filtering out other wavelengths as much as possible) provides that only the fluorescent signal can reach the camera, i.e. the excitation light and the light of the holographic light source are filtered out. In this embodiment, therefore, the apparatus comprises an emission filter 28 transmitting the fluorescent emitted light and being arranged on the common optical axis between the fluorescence detector 14 and the coupling-in element 20.
The illumination, having a coherence length required for the holographic system, is exemplary provided by a laser. The wavelength of the laser should be chosen such that it does not disturb the fluorescence measurement, and that it can be appropriately separated from the fluorescent illumination by means of a suitable filter, i.e. in the embodiment of Fig. 1 a first filter 30 transmitting the light of the holographic light source 16 (and for filtering out other wavelengths to the greatest possible extent) and being arranged between the holographic detector 18 and the sample holder spatial region 10. The filter 30 provides that only the diffracted images of the objects illuminated by the laser is recorded by the camera and that the fluorescent excitation light is completely cut out. The holographic image of the sample is recorded, through a suitable optical arrangement, by a - preferably digital - high-resolution camera. By selecting the fluorescent objects in a manner applied in the method according to the invention (i.e. searching for them in the fluorescence image) the processing speed of the method and apparatus according to the invention is significantly increased, since it is not necessary to reconstruct all of the holographic images but only those that are of interest, i.e. which comprise the three-dimensional model of at least one fluorescent object according to the identification step carried out on the fluorescence image. This allows for applying a higher-resolution detector, higher magnification, and, consequently, a narrower field of view because the flow speed and the size of the volume to be analysed are no longer limited by the evaluation speed of (digital) holograms. The location of a specific fluorescent object in the field of view is shown by the fluorescent signal. Thereby the reconstruction needs to be done for the holographic image corresponding to the given object. The parameters of the reconstruction can preferably be determined by processing a reduced-resolution image before reconstructing the full-resolution one, more precisely than what is allowed by optimizing processing times in known approaches. Determining the reconstruction distance as accurately as possible is very important. If there is sufficient time for determining the reconstruction distance, the image of the reconstructed objects can be of much higher quality. The method according to the invention allows for that. The support of the object (the region encompassing the object) can also be determined more precisely by applying the method according to the invention, since the processing time available for reconstructing certain really interesting objects can be increased by as much as an order of magnitude. Samples subjected to fluorescence analysing are typically so sparse that, when the sample is fed to the sample holder in a flow- through regime, a fluorescent object will be found in every tenth image or even more rarely. The time available for reconstructing the selected holographic images increases due to this ratio. If a sample is populated by fluorescent objects more densely than this, it can be diluted to allow for more efficient processing and analysis. However, because the method and apparatus according to the invention are e.g. applied for verifying the efficiency of water purification (for determining the number of fluorescent objects remaining there in a given purified sample), by the nature of the application usually a low number of fluorescent objects will be present in each sample. A fluorescent dye is applied for verifying that the purification process has killed all living organisms.
In an embodiment of the method according to the invention, after recording at least two fluorescence images, a differential fluorescence image is generated by subtracting the average of the fluorescence images (i.e. the average of the current and the earlier image) from the last recorded (i.e. current) fluorescence image, and carrying out the fluorescent object identification step on the differential fluorescence image. In this embodiment the background fluorescence is preferably averaged out by subtracting the average calculated using the earlier fluorescence images, and thus only the deviation from the average is detected. Thereby, because their signal is time-independent, the permanent fluorescent reflections, bubbles, cracks and camera faults are automatically eliminated with subtracting the average.
In the fluorescent object identification step of the method a threshold value (of light intensity measured in the fluorescence image) is preferably set below which the light intensity values detected during the search are not regarded as corresponding to any fluorescent object. The search for fluorescent objects comprises finding the maximum-intensity location in the fluorescence image. If such a value is found and it is larger than the predetermined threshold, then the measured values are examined in its vicinity, as they may also belong to the given object. Thereby, multiple detections of the same object can be ruled out because the regions belonging to a given object can be excluded from the subsequent search for further fluorescent objects. The search is preferably performed in a pixel-by-pixel manner, examining the light intensity recorded at the given pixel. If side-by-side objects are examined (which can be spatially situated behind each other) that would be detected as a single fluorescent object in the fluorescent object identification step, the situation is clarified during the holographic reconstruction step, which allows for making a decision on how the low-resolution fluorescence image should be interpreted and segmented. With sparse samples, though, such a "constellation" of fluorescent objects occurs very rarely.
The 2D-position of an object inside the analysed sample holder spatial region (measurement cell) is detected by means of the fluorescence camera. In the event of more than one objects being reconstructed in the analysed region based on the holographic image (e.g. at different depths) it has to be determined which object exhibited fluorescence and which did not do so. This can be achieved by the comparison and simultaneous analysis of the two measurements according to the above: the fluorescent objects are found essentially on the basis of the fluorescence image (and accordingly the non-fluorescent objects can be ignored). It may also happen that a fluorescent object is spatially extended, in which case according to the invention information provided by the two images can expediently be compared (by way of example, it can be established whether the contents of both images suggest an extended object).
By applying a threshold value the fluorescent objects can be differentiated from background noise. Because non-moving objects are eliminated by averaging the images and subtracting the average from the current image, preferably only the moving fluorescent objects (being moved inside the flow-through cell or having own locomotion) are shown in the differential fluorescence image resulting from the subtraction. The applied threshold value depends on the fluorescent signal (if the signal is weak, the threshold is low). Accordingly, the threshold value should be adjusted to match the currently applied illumination. It has to be noted that in the case of FDA staining (when the objects are made artificially fluorescent by applying a dye), where background fluorescence is increasing during the measurement, this variation should be taken into account when the threshold value is chosen, i.e. a time-variable threshold value may be needed. Thereby, in this embodiment, a threshold value being variable depending on the background fluorescence of the specific sample volume is applied. When auto-fluorescence is measured, background fluorescence does not change, but it still has to be analysed (because it can also be the result of infiltrating light rather than fluorescence). The threshold value is dependent on the illumination, on the sensitivity and dynamic range of the camera, the gain of the apparatus, the signal- to-noise ratio and signal strength. Any one of these parameters may change by way of example in case a component is replaced.
The timing of recording the images is illustrated in Figs. 2A-2C also referred to above. The fluorescence images and the holographic images are recorded in a time-overlapping manner. This is illustrated in Figs. 2A and 2B, showing the exposure time TF, and the exposure time TH corresponding respectively to the fluorescence image and the holographic image. Figs. 2A and 2B are intended to be interpreted together. They show the relationship of the typical length of the exposure times TF and TH and also illustrate that the exposure of the two images is started roughly at the same time, thus preferably providing for their overlap in time. During the relatively long exposure time TF the holographic image can be recorded with different timing, too. A much longer exposure time is typically applied for the fluorescence image, but because the required camera resolution is lower the combined exposure and readout times are typically the same for the two images (for the fluorescence image the exposure time is long but readout is fast, and conversely, for the holographic image the exposure time is short but image readout takes longer due to the larger resolution). Because the objects can get displaced during the long exposure period, a slight motion blur may occur. This, however, can be neglected due to the low resolution, i.e. it essentially does not affect the ability to identify the specific object in the holographic image.
As it will be described in the following, the occurrence of motion blur can also be exploited in a preferred embodiment (illustrated in Fig. 2C). In this embodiment, during a time portion TF2 of an exposure time TF of the fluorescence image the specific analysed sample volume is illuminated with two fluorescent excitation lights of different wavelengths, and, during another time portion TFi of the exposure time TF, the specific analysed sample volume is illuminated with only one of the two fluorescent excitation lights, the fluorescence image is recorded by imaging the fluorescent emitted light emitted under the effect of the fluorescent excitation lights, and in case at least one fluorescent object has been identified in the fluorescent object identification step, a fluorescent object excited by the fluorescent excitation lights applied during both of the time portions is identified based on the displacement of the at least one fluorescent object on the fluorescence image, and/or a fluorescent object excited by the fluorescent excitation light applied during either one of the time portions is identified.
Such an illumination scheme is illustrated in Fig. 2C, showing also the full exposure time TF; in the case depicted in the figure only one excitation illumination is applied in the time portion TFi (operating for example only the excitation light source 12a), with the other excitation being turned on for the time portion Tf2 ', thereby the currently analysed sample volume is illuminated with two excitation lights with different wavelengths. It is also conceivable that two illuminations are applied simultaneously during a first time portion which is followed by applying only one illumination in a second time portion, when one of them is switched off. The steps of the method according to this embodiment can also be carried out on an image recorded in the above manner.
This embodiment can therefore be applied for differentiating between fluorescent signals (for example, the red fluorescence of chlorophyll at 680 nm induced by blue illumination at 470 nm from the red fluorescence of phycocyanin at 650 nm induced by amber illumination at 595 nm) that otherwise cannot be differentiated b means of the fluorescence detector (camera) without a special dedicated filter. In the example the cells that also comprise phycocyanin, which have a special motion blur pattern, (the motion blur corresponding to the two objects will be different because the motion recorded in the image are of different duration, and thus the motion blur representing the displacement during the exposure period is also different) can be differentiated by turning on the amber illumination at an appropriate time. In Figs. 3A-3B exemplary corresponding fluorescence and holographic images are illustrated, while in Fig. 3C the reconstruction of the holographic image of Fig. 3B is shown. In Fig. 3A three objects appear, i.e. during the corresponding analysis three fluorescent objects were situated in the analysed sample holder spatial region at the time the image was recorded. In this specific example, green and red fluorescence can be discerned based on the fluorescence image (the red fluorescence originating in the example from auto-fluorescence, while the green one originating from dye-induced fluorescence), the image of the leftmost object being red and the images of the other two objects being green. The respective object images are indicated by arrows. Fig. 3B shows the holographic image corresponding to the fluorescence image shown in Fig. 3A. As shown in Fig. 3B, in addition to the images of the fluorescent objects several other object images (i.e. the interference patterns corresponding to them) appear. A great advantage of the invention is that by applying the method according to the invention the images corresponding to the fluorescent objects can be selected from the many object images that can be seen in the holographic image of Fig. 3B. As it is apparent from the comparison of Figs. 3A and 3B, based on the position of the object images of the fluorescence image, the object images corresponding to the fluorescent objects can selected from the object images comprised in the holographic image.
Fig. 3C shows the reconstruction of the object image (interference pattern) enclosed in the white box in Fig. 3B, in the figure the reconstructed shape of the fluorescent object can be observed. In the reconstructed image shown in Fig. 3C, ring-shaped artefacts can be seen because of the twin-image formed due to the inline arrangement applied for recording the holographic image. The disturbance caused by the twin-image can be reduced applying known approaches, the morphological analysis, i.e. the examination of the three-dimensional model obtained for the fluorescent object is not affected by the rings (the twin-image elimination process is slow, so it is not applied).
Fig. 4 shows a fluorescence image recorded with a relatively long exposure time. Due to the displacement of the fluorescent object the object image "leaves a trail", i.e. it exhibits motion blur. According to the above, through modifying the fluorescent excitation in an appropriately timed manner this may allow for differentiating objects that fluoresce at the same wavelength.
To avoid a large-scale motion blur, flow-through samples preferably have very low flow speeds during the exposures (the small-scale motion blur that is discussed above and can be preferably exploited is typically present even with stationary samples). After recording the images, however, the flow speed of the fluid can be increased applying a pump, followed by reducing the flow speed again for the next exposure. Images are therefore preferably exposed after the sample (set in motion by a pump) has slowed down or halted. The time required for that is not long relative to the applied exposure and readout times, so it does not significantly slow down the operation of the apparatus, i.e. images can preferably be recorded in relatively rapid succession. In an exemplary embodiment of the apparatus an exposure is taken every second, but a rate of 3-4 Hz may probably be achieved (in that case it can also be advantageously exploited that the exposure time of the fluorescence image and the readout time of the holographic image are roughly the same, and thus the two processes can be coordinated).
Known apparatuses wherein the same detector is applied for detecting both image types are not capable of such an operating mode. For measuring the fluorescent signal a long exposure time, high gain and a long illumination period are required, while the holographic image requires high resolution and short exposure times (and optionally also colour imaging). Thereby, in known apparatuses these procedures cannot be performed simultaneously but only in a sequential manner. In such cases, however, object identification can only be carried out if the objects are stationary or they move very slowly. If also super-resolution has to be applied in such known apparatuses, as it is often required with lens-less systems, then the processing speeds are reduced further, making it even more difficult to register the fluorescence and holographic images. In addition to that, known apparatuses are unable to fulfil the requirement that the holographic system (in the case of a lens- less setup) provides favourable resolution for small object-to-detector distances, while the dichroic mirror (essentially an interference filter capable of providing high optical density/suppression) that is applied for filtering out the fluorescent excitation light and is indispensable for such filtering in this setup, is usually 2-3 mm-thick due to its manufacturing technology. Accordingly, object-to-detector distances smaller than this distance is not possible in such known approaches.
Preferably, flow speed has to be adjusted such that any given object is recorded in only one hologram, but it should not exhibit motion blur in it, i.e. it should get displaced by an amount smaller than the pixel size during the exposure period. At the same time, in order make sure each object is imaged only once it has to be ensured that the entire analysed volume is replaced between two subsequent exposures.
In an embodiment of the method according to the invention a flow-through moving sample is applied and, after a specific sample volume has been replaced by another one, the steps of the method are carried out on at least one another specific sample volume. If the exposure time of the fluorescence detector is e.g. 80 ms (the time practically required for gathering enough light for accurate exposure, but this depends on the camera and the fluorescent illumination), it is desirable that the object is displaced at most by a few pixels during this period of time. With a displacement of 5 pixels, the displacement appearing on the holographic detector - if the resolution of the fluorescence detector is by way of example 256x256, and the resolution of the holographic detector is 2048x2048 - will be 8 times that (2048/256=8), which amounts to 40 pixels/80 ms. By way of example, the flow speed of the sample has to be reduced to that value. Since the holographic images are recorded applying short exposure times (on the order of 1 ms) such speeds will cause only negligible motion blur. After the image is recorded, the sample can be moved further, even at high speeds as that does not affect the analysed volume.
The holographic detector has an exemplary resolution of 10-15 megapixels (of which a region of 2048x2048 pixels is utilized as mentioned above); with the average detector size being 5x5 mm. Images of such size can be typically downloaded to a computer approximately at a rate of 4-5 frame/sec; the download time of a single image being approximately 200 ms, but it may be as high as 500 ms. Although cameras much faster than that do exist, these are very expensive and the processing of the obtained images and holograms is much slower, i.e. it is not worth applying such cameras. The holographic exposure is carried out for example by flashing lasers, with a duration in the order of milliseconds. Therefore the objects or their holograms seem to be stationary during the exposure, not moving even as much as a single pixel (there is no motion blur that would spoil the resolution). The fluorescence detector has low resolution (for example, 1-2 megapixels), and due to the differences in magnification only a small portion of it is utilized (e.g. 300x300 pixels), with the detector size being typically 5x5 mm. According to the above, relatively long exposure times and high gain values are to be applied in order that a sufficient amount of light can be gathered from the weak fluorescent signal. Of course in this case the applicable exposure time is also dependent on the intensity of the fluorescent excitation light (with shorter exposure times corresponding to strong illumination). In accordance with the above, for fluorescence images exposure times in the range of 40-200 ms are typically applied. With longer exposure times the objects could get displaced to a larger extent during the exposure, which may hinder accurate positioning, but with short exposure times this effect is practically negligible. For fluorescent object differentiation based on motion blur appearing in the fluorescence images (see Fig. 2C) it is appropriate to apply such exposure times.
In an exemplary case, therefore, a fluorescence image of 256x256 pixels and a holographic image of 2048x2048 pixels are recorded. The size of the utilized detector region also depends on the magnification/demagnification value of the projections. The difference between detector resolutions may be even greater than this, but that may negatively affect the estimation of object locations in the fluorescence image. With sparse samples, where the objects are separated by larger distances and thus are not overlapping irrespective of their fluorescence, even still greater resolution differences can be applied. Applying appropriate filters it is also possible to simultaneously record multiple different fluorescent signals. For that it has to be provided that the applied illuminations (excitation lights) do not interfere with one another, and that the emitted wavelengths can somehow be separated from one another. For example, two respective illuminations at 470 nm and at 600 nm may be applied (with 10 nm band-pass filters being applied as excitation filters). The applied emission filter transmits light above 500 nm (thereby the excitation light with a wavelength of 470 nm is prevented from reaching the fluorescence detector), and a band-stop filter (a so called 'notch filter') with a bandwidth of 20 nm is applied for filtering out the 600- nm illumination such that also that cannot reach the fluorescence detector. Thereby it is possible to detect samples dyed with FDA (emitting light at ~520 nm, for alive or dead analysis), the auto-fluorescence of chlorophyll B (emission at ~680 nm), phycocyanin emission (emission at ~650 nm, blue algae) and also phycoerythrin (emission at 570 nm, blue and red algae), while the holograms are for example recorded applying light with a wavelength of 405 nm. In the apparatus, fluorescent illuminations are preferably (i.e. also in the embodiment according to Fig. 1 ) provided obliquely, at low angles. This arrangement allows for the simultaneous application of multiple fluorescent lights, and thus the requirements for the applied dichroic filters (in the embodiment shown in Fig. 1 the coupling-in element 20 may function as a dichroic filter for fluorescent emitted lights, because it preferably transmits to the fluorescence detector only these lights) can be relaxed. In other words, if the fluorescent illumination reaching face to face the detector, the suppression provided by the dichroic filter should be much greater than in the case where the sample is illuminated obliquely, since in the latter case only scattered light has to be filtered out. Since each of the fluorescent illuminations are implemented as separate modules (the excitation light sources 12a, 12b), their appropriate focusing and bandwidth filtering can be provided, i.e. the emission filter (which in the embodiment of Fig. 1 is the emission filter 28) can be also optimized more easily. If another arrangement was applied, the merging and focusing of the illuminations would be complex, costly, and not really efficient optically. With the solution according to the present embodiment it is not necessary to apply multiple low-pass or high-pass mirrors. Such mirrors are expensive, have an efficiency of only about 90% and so may cause significant light loss, and would deteriorate the quality of both the fluorescence and the holographic imaging as they would include more refracting surfaces in the arrangement.
An afocal optical arrangement can be applied preferably for recording the holograms. The great advantage of such arrangements, compared to others, is that they provide, constant magnification for different objects inside the system, (i.e. the magnification is not dependent on object position), even if the illumination does not provide plane-wave reference. Thereby magnification calibration is not necessary. The essential feature of the applied arrangement is that instead of applying a lens (objective) for achieving the appropriate magnification, a lens system essentially arranged as a telescope is applied. This is preferable because, although with holographic reconstruction the single-lens system generates a magnification independent of object distance, it does so only if the illumination is parallel. If this condition is not fulfilled, the magnification of reconstructed objects will vary with distance (it can be calibrated, but not in a simple manner). However, with an afocal arrangement the magnification will be practically independent of the illumination parameters. This typically does not affect object segmentation, because in our case that is performed applying the optimal object reconstructions. The parameters of optimal reconstruction are established by comparing a number of reconstructions carried out at different distances (of course, calculating only lower- resolution images), applying special focus measures. During carrying out the method according to the invention preferably there is also time for determining the distance corresponding to the given reconstruction in a more accurate manner.
A dichroic mirror 31 can also be preferably arranged in the apparatus according to the invention. The dichroic mirror is adapted for diverting the fluorescent signal reflected from the holographic imaging lens (microscope objective). The dichroic mirror 31 is not arranged perpendicular to the common optical axis but slightly inclined with respect to it. Otherwise this signal would also enter the fluorescence camera and would generate an extra image recorded from the other side (but more blurry and defocused) of the measured volume. The filter 30 and the dichroic mirror 31 can preferably be replaced by a single optical element arranged at the position of the dichroic mirror 31 . In this case the filter 30 can be omitted.
In the holographic arrangement of the apparatus, e.g. an in-line arrangement is applied, i.e. the reference beam, too, is provided by the illumination. Such an arrangement is preferably applied in case the analysed sample is sparse, i.e. scattering on the measured objects can be treated as small perturbations of the reference beam. The advantages of this arrangement are simplicity and robustness (insensitivity to vibrations). Other arrangements may also be considered (namely, on-axis in-line and off-axis arrangements) but in such arrangements it has to be provided that the reference beam can be directed to the holographic detector along a separate path. These arrangements can be applied for analysing larger-sized, less translucent samples as well, and object phase reconstruction is also easier (simpler and faster numeric methods can be utilized) but they involve more complex optical apparatuses and are very sensitive to vibrations. It is expedient to make efforts to make the wavefront applied in the holographic arrangement regular (such that, if possible, it can be closely approximated by a plane wave). This can be provided by applying a single-mode laser wherein the emitting surface is preferably small, since a more extended emitting surface could restrict the achievable resolution.
In the apparatus according to the invention the coupling-in element is for example implemented by a dichroic low-pass mirror adapted for reflecting short-wavelength light and transmitting longer wavelengths. In our apparatus it is adapted for providing coherent illumination for the holographic arrangement, and for directing it first to the sample and then onto the holographic detector (the light of the holographic illumination being reflected by the dichroic low-pass mirror), while it transmits the fluorescent emitted light, which results in that the fluorescence camera imaging can be performed through it. The dichroic low-pass mirror is preferably arranged in a manner also shown in Fig. 1 , at an angle of 45° with respect to the common optical axis.
Some of the above described features of the apparatus, particularly the application of imaging lenses and filters and the resolutions and sensitivity values preferably applied in the detectors, can be applied in the method according to the invention independent of other features of the apparatus.
The method and apparatus according to the invention is capable of combining the fluorescence measurement capability of flow cytometers with the advantages provided by holographic (preferably digital holographic) images. By the preferred application of digital holographic microscopy, it is capable of reconstructing high- resolution images (applicable also for object classification) of the identified objects by selecting - based on the fluorescence image - the fluorescent objects from among the few dozen objects comprised in the image recorded by the holographic microscope. The invention is, of course, not limited to the preferred embodiments described in details above, but further variants, modifications and developments are possible within the scope of protection determined by the claims.

Claims

1 . A method for fluorescence analysis of a sample, comprising the steps of
- illuminating a specific analysed sample volume of the sample with fluorescent excitation light and recording a fluorescence image by imaging fluorescent emitted light of the specific analysed sample volume emitted under the effect of the fluorescent excitation light, and
- recording, overlapping in time with recording the fluorescence image, a holographic image of the specific analysed sample volume,
c h a r a c t e r i s e d by carrying out a fluorescent object identification step on the fluorescence image, and, in case at least one fluorescent object has been identified in the specific analysed sample volume in the fluorescent object identification step,
- assigning, on the fluorescence image, at least one object position to the at least one fluorescent object, respectively, and
- generating, by reconstructing the holographic image, a reconstructed holographic image, and identifying on the reconstructed holographic image a reconstructed model of the at least one fluorescent object based on the at least one object position.
2. The method according to claim 1 , characterised by applying a flow-through moving sample and, after a specific analysed sample volume has been replaced by another one, carrying out the steps of the method on at least one another specific analysed sample volume.
3. The method according to claim 2, characterised by generating, after recording at least two fluorescence images, a differential fluorescence image by subtracting the average of the fluorescence images from the last recorded fluorescence image, and carrying out the fluorescent object identification step on the differential fluorescence image.
4. The method according to claim 3, characterised by applying, in the fluorescent object identification step, a threshold value for identifying a fluorescent object.
5. The method according to claim 4, characterised by applying a threshold value being variable depending on a background fluorescence of the specific sample volume.
6. The method according to any of claims 1 to 5, characterised by
- illuminating, during a time portion (TF2) of an exposure time (TF) of the fluorescence image, the specific analysed sample volume with two fluorescent excitation lights of different wavelengths, and, during another time portion (TFi ) of the exposure time (TF), illuminating the specific analysed sample volume with only one of the two fluorescent excitation lights,
- recording the fluorescence image by imaging the fluorescent emitted light emitted under the effect of the fluorescent excitation lights, and
- in case at least one fluorescent object has been identified in the fluorescent object identification step, identifying a fluorescent object excited by the fluorescent excitation lights applied during both of the time portions (TFi , TF2) based on the displacement of the at least one fluorescent object on the fluorescence image, and/or a fluorescent object excited by the fluorescent excitation light applied during either one of the time portions (TFi , TF2).
7. The method according to any of claims 1 to 6, characterised in that an exposure time (TF) of 40-200 ms is applied for recording the fluorescence image, and/or an exposure time (TH) of 0.5-5 ms is applied for recording the holographic image.
8. An apparatus for fluorescence analysis of a sample, the apparatus comprising
- a sample holder spatial region (10) adapted for receiving a specific analysed sample volume of the sample,
- one or more excitation light sources (12a, 12b) adapted for illuminating the sample holder spatial region (10) with fluorescent excitation light,
- a fluorescence detector (14) adapted for recording a fluorescence image by imaging the fluorescent emitted light of a specific analysed sample volume emitted under the effect of the fluorescent excitation light, the specific analysed sample volume being in the sample holder spatial region (10) during an analysis, and
- a holographic arrangement adapted for recording holographic images of the specific analysed sample volume, wherein the holographic arrangement comprising a holographic light source (16) and a holographic detector (18), c h a r a c t e r i s e d in that
- the fluorescence detector (14) and the holographic detector (18) are arranged on a common optical axis with the sample holder spatial region (10) at opposite sides of the sample holder spatial region (10), and
- it comprises a coupling-in element (20) adapted for imaging an object wave originating from the holographic light source (16) onto the common optical axis and directing it towards the sample holder spatial region (10), and also adapted for reflecting the light of the holographic light source (16) and transmitting the fluorescent emitted light.
9. The apparatus according to claim 8, characterised by comprising a fluorescence-imaging lens (22) arranged on the common optical axis between the fluorescence detector (14) and the coupling-in element (20).
10. The apparatus according to claim 9, characterised in that a depth of focus of the fluorescence-imaging lens (22) is adjusted, by adjusting the demagnification/magnification value thereof, to a dimension of the sample holder spatial region (10) along the common optical axis.
1 1. The apparatus according to claim 9 or claim 10, characterised in that the demagnification/magnification of the fluorescence-imaging lens (22) is in the range of 0.5-1.5.
12. The apparatus according to any of claims 8 to 1 1 , characterised by comprising a holographic imaging lens (24) arranged between the sample holder spatial region (10) and the holographic detector (18).
13. The apparatus according to claim 12, characterised in that the magnification of the holographic imaging lens is in the range of 1.5-10.
14. The apparatus according to any of claims 8 to 13, characterised by comprising, arranged along the common optical axis:
- an emission filter (28) transmitting the fluorescent emitted light and arranged between the fluorescence detector (14) and the coupling-in element (20), and/or
- a first filter (30) transmitting the light of the holographic light source (16), and arranged between the holographic detector (18) and the sample holder spatial region (10).
15. The apparatus according to any of claims 8 to 14, characterised by comprising an excitation filter (26a, 26b) transmitting the fluorescent excitation light, and arranged between the excitation light source (12a, 12b) and the sample holder spatial region (10).
16. The apparatus according to any of claims 8 to 15, characterised in that the excitation light source (12a, 12b) is arranged such that it is capable of illuminating the sample holder spatial region (10) from a direction different from the common optical axis.
17. The apparatus according to any of claims 8 to 16, characterised by comprising more than one excitation light sources (12a, 12b) adapted for emitting excitation light of wavelengths being different from each other.
18. The apparatus according to any of claims 8 to 17, characterised in that the resolution of the fluorescence detector (14) is 0.5-3 megapixels/25 mm2 and/or the resolution of the holographic detector (18) is 5-20 megapixels/25 mm2.
PCT/HU2016/000078 2016-01-25 2016-12-08 Method and apparatus for fluorescence analysis of a sample WO2017130009A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
HUP1600037 2016-01-25
HUP1600037A HU231357B1 (en) 2016-01-25 2016-01-25 Method and apparatus for analysing fluorescence of a sample

Publications (1)

Publication Number Publication Date
WO2017130009A1 true WO2017130009A1 (en) 2017-08-03

Family

ID=89992060

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/HU2016/000078 WO2017130009A1 (en) 2016-01-25 2016-12-08 Method and apparatus for fluorescence analysis of a sample

Country Status (2)

Country Link
HU (1) HU231357B1 (en)
WO (1) WO2017130009A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210187503A1 (en) * 2019-12-19 2021-06-24 Personal Genomics Taiwan, Inc. Apparatus and system for single-molecule nucleic acids detection

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7158220B2 (en) * 2018-09-11 2022-10-21 浜松ホトニクス株式会社 Measuring device and method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5162990A (en) * 1990-06-15 1992-11-10 The United States Of America As Represented By The United States Navy System and method for quantifying macrophage phagocytosis by computer image analysis
WO1998050577A1 (en) * 1997-05-05 1998-11-12 Chemometec A/S A method and a system for determination of somatic cells in milk
US20080055595A1 (en) * 2006-08-30 2008-03-06 Olson David C System and method of capturing multiple source excitations from a single location on a flow channel
US20130070251A1 (en) * 2011-09-16 2013-03-21 University Of Massachusetts Systems and Methods of Dual-Plane Digital Holographic Microscopy
US20140376816A1 (en) * 2011-12-07 2014-12-25 Katholieke Universiteit Leuven, KU LEUVEN R&D Analysis and Sorting of Objects in Flow

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5162990A (en) * 1990-06-15 1992-11-10 The United States Of America As Represented By The United States Navy System and method for quantifying macrophage phagocytosis by computer image analysis
WO1998050577A1 (en) * 1997-05-05 1998-11-12 Chemometec A/S A method and a system for determination of somatic cells in milk
US20080055595A1 (en) * 2006-08-30 2008-03-06 Olson David C System and method of capturing multiple source excitations from a single location on a flow channel
US20130070251A1 (en) * 2011-09-16 2013-03-21 University Of Massachusetts Systems and Methods of Dual-Plane Digital Holographic Microscopy
US20140376816A1 (en) * 2011-12-07 2014-12-25 Katholieke Universiteit Leuven, KU LEUVEN R&D Analysis and Sorting of Objects in Flow

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GABRIEL BIENER ET AL: "Combined reflection and transmission microscope for telemedicine applications in field settings", LAB ON A CHIP, vol. 11, no. 16, 1 January 2011 (2011-01-01), pages 2738, XP055299022, ISSN: 1473-0197, DOI: 10.1039/c1lc20169g *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210187503A1 (en) * 2019-12-19 2021-06-24 Personal Genomics Taiwan, Inc. Apparatus and system for single-molecule nucleic acids detection

Also Published As

Publication number Publication date
HU231357B1 (en) 2023-03-28
HUP1600037A1 (en) 2017-07-28

Similar Documents

Publication Publication Date Title
JP5160158B2 (en) Imaging system and method
CA2445044C (en) Method and apparatus for correcting crosstalk and spatial resolution for multichannel imaging
US8830476B2 (en) Methods and apparatuses for contact-free holographic imaging of aerosol particles
US10930061B2 (en) Three-dimensional image processing to locate nanoparticles in biological and nonbiological media
CN107003229B (en) Analytical method comprising holographic determination of the position of a biological particle and corresponding device
US20060175561A1 (en) Particle shadow velocimetry
JP2009537021A (en) Laser irradiation system in fluorescence microscopy
JP2004507743A (en) Alternative detector configuration and operating mode of time delay integrating particle analyzer
CN108693098B (en) Cell counting analysis method for cell sample
US20180189963A1 (en) Method for counting and characterization of particles in a fluid in movement
EP3532822B1 (en) Trans-illumination imaging with use of interference fringes to enhance contrast and find focus
US20060238846A1 (en) Imaging device
US7217937B2 (en) Automatic identification of suspended particles
US20140022373A1 (en) Correlative drift correction
WO2017130009A1 (en) Method and apparatus for fluorescence analysis of a sample
EP3781977A1 (en) Depicting of objects
EP0626575B1 (en) Method and apparatus for microscopic imaging
JPWO2018081037A5 (en)
WO2023022091A1 (en) Analysis system, observation container, analysis method and program
US11933707B2 (en) Depicting of objects
US10866398B2 (en) Method for observing a sample in two spectral bands, and at two magnifications
Splitter et al. Glossary (Terms Are Defined with Respect to Confocal Imaging)
JP2023519423A (en) LENSLESS MICROSCOPY IMAGING SYSTEM AND METHOD AND BIOCHEMICAL DETECTION SYSTEM AND METHOD
Basiji Multispectral Imaging Flow Cytometry
CN114397752A (en) Imaging system and method for flow-type particle imaging

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16840328

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16840328

Country of ref document: EP

Kind code of ref document: A1