RED respond to Apple in compressed RAW patent battle

[newfoundmass]

4 minutes ago, Laser Blue said:

My understanding is that patents deemed improperly filed or invalid result in the patent concept becoming public domain and permanently unpatentable in perpetuity.

That would certainly explain why they would defend this with all their might and in ways that might come off to others as aggressive. It's the difference between losing that advantage now vs. When the patent ends and everything is fair game. 

[Laser Blue]

7 minutes ago, Andrew Reid said:

Thanks for that post, very interesting.

Were you aware in 2006 of the Russian Kinor camera with similar sensor to the RED ONE at the time?

Same question regarding Pavavision's development of the Genesis, or was that only started later on?

How about the very early digital cinema work with Lars Von Trier of Anthony Dod Mantle who pioneered digital cinematography in the industry. Peter Jackson didn't, in my opinion. Anthony Dod Mantle did. Was RED and Peter Jackson's approach novel in light of the cameras Anthony Dod Mantle used? I can't say as not a patent lawyer but food for thought.

How about the Sony Cine Alta range. F35, etc.?

Panavision Genesis? Early Panasonic stuff?

SiliconImaging?

Kinor?

I agree with you, Graeme Nattress and the team deserve credit for disrupting the market.

We want affordable compressed RAW and open source codecs to play nicely in the same industry alongside RED and they aren't keen on it happening, so that loses them some support from me, as I am, writing EOSHD and waiting for the Nikon Z6 raw firmware update.

What about Cinema DNG 3:1 as it is under the 6:1 minimum compression in the RED patent... Can't that pass without infringement, I wonder?

Magic Lantern 10bit lossless maybe?

Or even just plain old 1080p uncompressed RAW?!

Must be a way to give us something in the current cameras, without having to pay RED for the license to do so.

Interesting take, cheers.

Well, the F35 recorded to HDCam SR tape (we had an SR VTR, but used it only for finished master content ingest for its 12-channel surround  never as source media ingest,) Panavision Genesis was 1920x1080 (sub DCI 2k) resolution too low for high-CGI films, Panasonic DVX-100 and Sony CineAlta cams were also basically HD res...

In my view today, dCinema technologically = digital recording non-tape media, not film, 4K resolution and visually lossless. P2  was digital media but HD res, CineAlta was tape and HD res. Panavision was HD res...

As for Von Trier vs. Jackson pioneering, I defer to you.

And while I sympathize with ATOMOS, I want internal RAW recording.

[Emanuel]

Andrew is doing a terrific job... So, no one can open a camera website after him, isn't it?

As Anthony (squig) wrote here to follow a suggestion of mine that one of these days we all need to pay prior licensing if someone has the idea to register a patent on planet Earth, go figure if Nestlé demands any revenue from vanilla ice cream made by my wife at my kitchen, I'd dare to wonder myself.

Wait, that's actually Milk Chocolate coated *sigh* so maybe they'll forgive...?

[Shell64]

53 minutes ago, Cliff Totten said:

If there is ANYBODY on Earth that wants the RED pattent to hold up in court.....its Atomos.

If that pattent dies, everybody will do ProResRAW without Atomos.in camera.

Atomos wants to be the sole ProResRAW ecternal recording vendor....and they NEED this for their future.

Yeah. And I personally want the patient to die. Atomos has an unfair monopoly because of this. I don’t want to pay $500 to record raw over a crappy HDMI cable when my camera is capable of 4K RAW to the sd card. Magic lantern proves this. 

[Emanuel]

I actually wonder if who deals with IP in US has any clue what raw means?

I dunno why all these people have the habit to write RAW instead... Can anyone explain me please? : -D

[Andrew Reid]

3 minutes ago, Emanuel said:

I actually wonder if who deals with IP in US has any clue what raw means?

I dunno why all these people have the habit to write RAW instead... Can anyone explain me please? : -D

We know it's not an acronym but RAW looks cooler 

10 minutes ago, Shell64 said:

I don’t want to pay $500 to record raw over a crappy HDMI cable when my camera is capable of 4K RAW to the sd card. Magic lantern proves this. 

+10000000

[Otago]

11 minutes ago, Emanuel said:

I dunno why all these people have the habit to write RAW instead... Can anyone explain me please? : -D

Because "raw footage" is ambiguous but "RAW footage" isn't. 

[Andrew Reid]

Yes raw footage could mean any files straight off the card but people understand it better when you say RAW. Not perfect sense but it does the job.

[Video Hummus]

I don’t think anybody disputes REDs contribution to digital cinema. What is being discussed is their subsequent behavior revolving around a questionable patent (Apple thinks so anyway) that they use to suppress innovation and competition unfairly (if the patent isn’t valid of course).

Also if the patent is for compressed raw recording internally in camera how is Atomos infringing when they don’t sell cameras? Is RED claiming their patent covers any device that stores compressed RAW data from a sensor?

Maybe sigma thinks cinemaDNG to an external storage device is not infringement.

1 hour ago, EspenB said:

REDs continued existence relies almost entirely on the redcode patent.

I don’t think so. They have a great sensor, color science, and brand loyalty and following. Just like every other camera manufacturer without compressed RAW capability. ARRI being highly successful. People choose their cameras on many different aspects. 

[Andrew Reid]

If we can't have ProRes RAW we can at least have ProRes. Good enough for Arri!

[Shell64]

Why stop there?  How about 28 mbps avchd with line skipping!

[Andrew Reid]

8 minutes ago, Shell64 said:

Why stop there?  How about 28 mbps avchd with line skipping!

Good enough for Soderbergh!

[Skip77]

6 hours ago, Andrew Reid said:

In Apple's attempt to overturn RED's claims over visually lossless compressed raw video, the US Patent Office has published documents submitted by RED. These explain their side of the story with particular regard to REDCODE.

If RED can continue to prove that the approach to their codec was novel, RED will win and Apple will have to compensate RED or make a deal in order to sell ProRes RAW in our devices and cameras, such as the Nikon Z6.

Read the full article

I thought Atomos already has agreements with RED to record / capture Pro Res RAW thru their recorders?  or are you saying Apple would have to do the same-thing Atomos had to do with RED?

[Andrew Reid]

Atomos might have yes. But ask yourself why the Nikon Z6 firmware update still isn't out yet.

[Sun Daze]

Thank you Andrew for this.  Here's some of my analysis.

Here's from Apple's Cliff Reader:

Ex. 1001, Fig. 1.
37. As discussed below, none of these techniques or characteristics were new or non-obvious as of the ’299 patent’s earliest effective filing date. Video processing systems and methods similar to ’299 patent were known. For example, Presler (Ex. 1005) teaches a similar video camera system for processing and transmitting high resolution video data (Ex. 1005, Abstract) and Molgaard (Ex. 1006) teaches similar processing and lossless compression techniques, including compressing, decompressing, and demosaicing mosaiced image data (Ex. 1006, Abstract). In my opinion, the disclosures provided in Presler, Molgaard, and other prior art discussed below render obvious the claims that I have been asked to analyze in the ’299 patent.

A. “Raw Mosaiced Image Data”

45. This term is used in independent claims 1 and 15. The specification of the ’299 patent does not offer an express definition of “mosaiced” image data. Apart from the claims and Abstract, the ’299 patent does not use the term “mosaic” or “mosaiced,” and refers only to the terms “demosaic,” “demosaiced,” and “demosaicing.” See e.g., Ex. 1001, 8:23-26, 10:25-27. The term “mosaiced,” however, is well known in the art as evidenced by Long (Ex. 1008), which describes “raw” data as received directly from a camera or similar device:

When you shoot in raw format ... no demosaicing is performed by the camera. Instead, the raw data that your image sensor captures is written directly to your camera’s storage card. Demosaicing is then the performed in your raw conversion software.

Ex. 1008, p.33. Long also describes the process of creating raw “mosaiced” data: Each pixel on your camera’s sensor has a colored filter over it, usually
a red, green, or blue filter ... each pixel on the sensor is able to register
one primary color. To turn this mosaic of primary-color pixels into a full-color image, a process called demosaicing is employed.

Ex. 1008, p.32. Zhang (Ex. 1007) also describes this process: “primary color samples are interleaved in a two-dimensional (2-D) grid, or color mosaic, resembling a three-color checkerboard.” Ex. 1007, p.1.

VII. IDENTIFICATION OF HOW THE CLAIMS ARE UNPATENTABLE

51. It is my opinion that claims 1-9, 11-12, 14-24, 26-27, and 29 are obvious over U.S. Patent No. 9,565,419 to Presler (Ex. 1005) in view of U.S. Patent No. 7,656,561 to Molgaard (Ex. 1006), further in view of U.S. Patent No. 5,600,373 to Chui et al. (“Chui,” Ex. 1015).

52. It is also my opinion that claims 10 and 25 are obvious over Presler in view of Molgaard and Chui, and further in view of U.S. Patent No. 7,349,574 to Sodini (Ex. 1013).

53. It is also my opinion that claims 13 and 28 are obvious over Presler in view of Molgaard and Chui, and further in view of U.S. Patent No. 8,170,402 to Frost-Ruebling (Ex. 1014).

A. Claims 1-9, 11-12, 14-24, 26-27, and 29 are obvious over Presler in view of Molgaard, further in view of Chui

1. Summary of Presler

54. Similar to the video camera described in the ’299 patent, Presler describes a “portable digital camera and recording system.” Ex. 1005, 4:35-36. Presler’s portable digital camera and recording system is similarly designed for use of applications with “high definition raw images at film or video rates for HD, 2K and 4K, cinema quality production.” Ex. 1005, 6:21-22.

55. Presler’s portable digital camera and recording system includes a mobile docking camera with a docking camera module, HD/2K/4K sensor unit, a modular processing subsystem, and removable digital storage, as shown in Figure 8 below.

56. The mobile docking camera of Presler includes an optical assembly 22

which allows light to pass into the portable housing of the camera and onto the HD/2K/4K sensor unit 30. Ex. 1005, 6:13-16, 17:5-8. The sensor unit 30 of Presler may include color filters for capturing raw mosaiced image pixel data. See Ex. 1005, 6:32-34, 14:37-39, 16:16-21.

57. Image processing of the raw image data, including compression, is performed by the frame buffer system 32 and the processing system 34 of Presler’s camera. See Ex. 1005, 8:23-28, 13:53-57. Compressed image data may be stored in the removable digital storage 44 within the mobile docking camera of Presler. Ex.

1005, 12:13-15. The mobile docking camera is also configured to transmit “2K raw image data at fifty (50) images per second with 10-bit per pixel data.” Ex. 1005, 10:18-21.

58. I have been notified that Presler claims and is entitled to the filing date of Provisional Application No. 60/923,339 (Ex. 1012, “the ’339 Application”) filed on April 13, 2007. See Ex. 1005, 1:8-13; Ex. 1012. Petitioners note that the number of the ’339 Application is incorrectly recorded in Presler as 60/923,399. Ex. 1005, 1:8-13.

2. Summary of Molgaard

59. Molgaard teaches image processing techniques similar to those described in the ’299 patent, as well as image processing similar to that performed by the digital camera of Presler. For example, Molgaard teaches “lossless, near- lossless, and lossy compression and decompression of digital image data.” Ex. 1006, Abstract. Further, the processing method of Molgaard “is optimized for raw image data from a sensor with a Bayer filter pattern.” Ex. 1006, Abstract. An example of Molgaard’s processing system is shown in Figure 2B below.

Molgaard further teaches that raw mosaiced image data received by an image sensor 14 includes a single data value for each color element of a Bayer filter. See Ex. 1006, 8:26-40. An example of this pattern is shown below in Figure 9.

Ex. 1006, Figure 9.

61. Molgaard’s processing method also includes black calibration where raw mosaiced image data is processed to remove fixed pattern noise:

In the embodiments outlined by the block diagrams of FIGS. 2A and B,

raw image data can be black calibrated to remove the fixed pattern noise which is generated as a function of temperature as well as integration time in the image sensor array. This process is called black calibration as the fixed pattern which is subtracted is obtained while the image sensor is not illuminated.

1006, 10:27-30.

3. Summary of Chui

63. Chui is directed to “method and apparatus for performing video image compression and decompression.” Ex. 1015, Abstract. Chui also lists “[e]xamples of conventional lossless data compression techniques” including “Huffman encoding, Fano-Shannon encoding, and arithmetic encoding, as well as dynamic variations of the Huffman and Fano-Shannon probabilistic encoding techniques” and includes a lossless compressor that can implement Huffman encoding. Ex. 1015, 1:58-62, 17:62-18:3.

4. Reasons to Combine Presler, Molgaard, and Chui

64. In my opinion, a POSITA would have found it obvious to combine the teachings of Presler, Molgaard, and Chui. All three references disclose image acquisition from a camera, and subsequent image processing operations, notably including image compression. For example, all three references discuss the JPEG image compression standard. See Ex. 1005, 22:56-66; Ex. 1006, 10:45-48; Ex. 1015, 2:16-24. In particular, a POSITA would have been motivated to combine Molgaard’s image processing techniques with Presler’s video camera and processing system that also captures image data and includes similar image

processing functionality. A POSITA would have recognized that Molgaard’s techniques could have been easily used to improve Presler’s similar video camera and would have improved performance of the video camera, such as enabling faster sampling of imaging data. See Ex. 1005, 5:57-66; Ex. 1006, Abstract, 7:1-3.

65. A POSITA would have recognized that Molgaard’s image processing techniques would have been readily applied to Presler’s video camera. For example, both references describe using a single image sensor with a Bayer filter to generate Bayer image data. See Ex. 1005, 14:37-39; Ex. 1006, 8:30-33. Furthermore, both teach processing raw image data from the image sensor. See Ex. 1006, Figure 2B (annotated) (showing an input of “RAW image data” that is passed through a “Preconditioning” step (as annotated below).

66. Based on these and other similarities, a POSITA would have

understood that the pre-processing of raw image data, including Bayer image data in Molgaard would readily be applied Presler’s video camera that also captures and processes Bayer image data.

67. In particular, both Presler and Molgaard disclose systems and methods in which light is captured by an imaging sensor comprising a mosaiced array arranged according to the Bayer pattern, which is evidenced by U.S. Patent No. 3,951,065 to Bayer (Ex. 1016). And, Presler explicitly discloses Bayer sensors, with mosaiced pixel arrays. See Ex. 1005, 14:25-26, 14:37-39, 18:44-51, and

Figures 2-4, 6-8, 10-11. Molgaard similarly discloses capturing light with an image sensor comprising a mosaiced array arranged according to the Bayer pattern. Ex. 1006, Abstract, 8:26-40, 12:5-9, 13:17-31. Although Presler does not discuss the specific details of capturing light incident on the sensor or the construction of the Bayer mosaiced array (although referring to the image sensor 30 as “pixelated”), Molgaard discloses both the capture of light by the sensor and the sensor arrangement. See e.g., Ex. 1006, Abstract, 6:27-39, 8:26-30, 9:51-62, 12:7-8 and Figure 8.

68. A POSITA would have understood that when the art discusses outputting raw data, it is discussing the pixel data from each element of the Bayer pattern array. See e.g., Ex. 1005, 10:37-38; Ex. 1006, 15:30-33. In this pattern, each pixel captures light of only one primary color. To form a complete image with all three colors for each pixel, the raw data must be spatially interpolated. This is evidenced by Presler which states: “n recent years, many digital still cameras or dual-mode video and still camcorders have also been developed which use single image sensors with color filter arrays. ... These digital still cameras and camcorders use dedicated hardware functions or targeted function digital signal processors (DSP) to perform image processing to interpolate and colorize the raw image data from the image sensor.” Ex. 1005, 2:35-50. Likewise, Molgaard states: “[r]aw image data means data containing only pixel values that have been read out

from the image sensor. The data may be amplified and converted, but no new pixel values have been added to the data e.g. by color interpolation.” Ex. 1006, 6:27-30. Thus, both Presler and Molgaard disclose capturing images using pixelated sensors with the Bayer mosaic pattern and outputting raw pixel data.

69. Although Presler may not explicitly disclose the use of Huffman coding to compress the processed image data into compressed processed image data, Presler discloses the use of Cineform compression. Ex. 1005, 21:64-22:3, 22:61-66, 24:54-60, 25:1-2, and Figure 12. A POSITA would have understood that Cineform employs Huffman coding for entropy coding the compressed data. Chui explicitly discloses Huffman coding either as a standalone lossless compression technique (Ex. 1015, 2:56-62, 17:66-18:3, 31:15-19) or in combination with lossy coding (Ex. 1015, 27:10-22). Both Presler and Chui are concerned with compression, thus a POSITA would have understood that Chui provides support for the disclosure of the use of Huffman coding by Presler. A POSITA would have understood that well-known lossy compression techniques, such as the Wavelet compression as disclosed by Presler (Ex. 1005, 13:46-57) comprise a sequence of operations in which the information in the image is first isolated and then entropy coded. Presler discloses the compression of individual video frames – i.e., images, can be performed using an image compression technique according to the JPEG2000 standard. See Ex. 1005, 22:61-66.

Presler also discloses use of the SATA interface for storage. See Ex. 1005, 12:13-23, 12:42-52, 15:55-61, 16:45-50, 22:4-9, 21:9-15. A POSITA would have known that the SATA specification supports storage at rates up to 3GB/s. See Ex. 1009, p.50. It would have been obvious to combine the specification of the SATA standard with the disclosed use of the standard, to support the disclosed capability of Presler’s system to store compressed processed images data at a rate of at least about 23 frames per second. See Ex. 1009, p.50.

71. Presler further discloses various system architectures for implementing the disclosed inventions including the use of hardware to perform compression. See e.g., Ex. 1005, 2:63-3:2 10:62-11:19, 13:30-57. A POSITA would have understood that before the earliest priority date of the ’299 patent in 2007, it was common practice to employ a compression chip as a hardware component of a digital video system. For example, set-top boxes, DVD players, and digital camcorders all used hardware compression chips as a component of complete digital video systems.

72. Second, a POSITA would have recognized that Molgaard’s data processing techniques would have been ideally suited for meeting the high-speed data requirements of Presler’s camera. See Ex. 1005, 4:33-39. For example, Molgaard’s compression algorithm “enables very fast subsampling of an already compressed image”—such as imaging data in Presler’s camera—to “improve

 

MY FINDING: 

It is not unique to combine Pressler with Molgard - it's rather obvious.  And with Chui, to be using jpeg to do this is pretty obvious.  Red did it and it's great, but you can't patent something that already exists and was already patented by combining 2 patents and calling it a day

 

Here's from Red's Response - 

"Finally, RED’s successful reduction to practice is corroborated by photographs, documentation, the testimony of current and former non-inventor employees who had direct personal knowledge and involvement with the RED ONE project, and the testimony of Academy Award winners Sir Peter Jackson and Steven Soderbergh. Indeed, with respect to their qualifications to assess the visually lossless"

While Boris and Natasha used lossy compression, the ’299 patent claims recite the use of Huffman compression. Nevertheless, a patent owner may antedate a reference even if there are differences between the patent owner’s evidence of actual reduction to practice and the challenged claim. Specifically, to the extent any such differences exist, the patent owner may antedate the reference “by showing that the differences between the claimed invention and the showing [of reduction to practice] . . . would have been obvious to one of ordinary skill in the art,” in view of the patent owner’s evidence. M.P.E.P. § 715.02 (9th Ed. Rev. 8, Jan. 2018) (emphases added); see In re Spiller, 500 F.2d 1170, 1178 n.5 (C.C.P.A. 1974) (Rich, J.). Such a showing is sufficient to antedate a reference because the patent owner’s reduction to practice “carries with it possession of variations and adaptations which would, at the same time, be obvious to one skilled in the art.” In re Spiller, 500 F.2d at 1178 n.5, 1176-78.

Huffman compression is a lossless compression technique that has been well known in the art for several decades. Nattress Decl. ¶ 30. For example, Petitioner’s exhibit 1015, a U.S. patent issued in 1997, refers to Huffman encoding as a 

“conventional lossless data compression technique[] . . ..” Ex. 1015 at 1:56-60. Petitioner’s declarant also argues that Huffman compression was a known technique whose implementation was within the level of ordinary skill in the art. Ex. 1003 at 60.

Although a lossless compression technique provides a lower compression ratio than lossy compression, this impacts the write speed and memory on the hard drive. Nattress Decl. ¶ 31. It does not lower the visual quality of the output. Id. Thus, because the REDCODE on Boris produced visually lossless output with a lossy compression technique, a person of ordinary skill in the art would know that REDCODE would successfully produce visually lossless output using a lossless compression routine like Huffman compression. Id. This result follows necessarily because a lossless compression technique removes any potentiality for compression artifacts caused by lossless compression. Id.

Accordingly, the reduction to practice of RED’s novel and nonobvious video camera capable of visually lossless output using lossy compression was tantamount to reducing to practice a video camera capable of visually lossless output using Huffman (i.e., lossless) compression. As established by Petitioner’s own reference and declarant, such a variation on the compression technique was conventional, and implementation of that conventional variation was well within the capability of those of skill in the art."

I still don't quite see their rebuttal.

 

 

[Jonesy Jones]

1 hour ago, Emanuel said:

I dunno why all these people have the habit to write RAW instead... Can anyone explain me please? : -D

I’ve noticed a differentiation between the use of RAW and raw. 

When people use RAW they, almost without fail, mean it how we are discussing it in this thread.

When people use raw they often mean straight out of camera, even one that doesn’t shoot RAW. Like, “here is some raw footage straight from my gh5.”

I know not everyone differentiates the way I do, but I wish they did. But at least when someone uses RAW you know exactly what they mean.

[ThadG]

Apple wouldn’t buy Red just for a patent to protect their ProRes Raw format. Pro software is a tiny fraction of Apple’s business and I guarantee they have no desire to get involved in the esoteric world of digital cinema camera manufacturing. It’d be a huge waste of money. If they can’t invalidate, they’ll strike a licensing deal. They can certainly afford it.

[EspenB]

53 minutes ago, Sun Daze said:

I still don't quite see their rebuttal.

Steven Soderbergh says so! That is their only defence. ?   

[Video Hummus]

Maybe Nikon and Atomos felt that RAW over HDMI isn’t RAW over SATA interface as described in ‘314 patent?

Kinda weird Nikon started selling video creator kits with a Ninja V right after they announced it and it’s been so long and not a word from either company.

Apple has lots of money. Why wouldn’t they just strike a licensing deal and move on? I think there is more to it than that.

And since Sony and Red settled out of court we don’t know what the agreement was. Maybe Sony lost and RED was facing their own infringement of Sony’s patents so they settled out of court and went their separate ways with NDAs in hand.

[Skip77]

1 hour ago, Sun Daze said:

Thank you Andrew for this.  Here's some of my analysis.

Here's from Apple's Cliff Reader:

 

MY FINDING: 

It is not unique to combine Pressler with Molgard - it's rather obvious.  And with Chui, to be using jpeg to do this is pretty obvious.  Red did it and it's great, but you can't patent something that already exists and was already patented by combining 2 patents and calling it a day

 

 

 

RED will lose this battle.  Apple has been down this road before and it says a lot the they are going after RED in the courts. 

Remember RED suing SONY?

22 minutes ago, Video Hummus said:

Maybe Nikon and Atomos felt that RAW over HDMI isn’t RAW over SATA interface as described in ‘314 patent?

Kinda weird Nikon started selling video creator kits with a Ninja V right after they announced it and it’s been so long and not a word from either company.

Apple has lots of money. Why wouldn’t they just strike a licensing deal and move on? I think there is more to it than that.

And since Sony and Red settled out of court we don’t know what the agreement was. Maybe Sony lost and RED was facing their own infringement of Sony’s patents so they settled out of court and went their separate ways with NDAs in hand.

Atomos has said ProRes RAW is coming later this year.   The screen capture from Atomos was dated Aug 1, 2019

Nikon also outlined what the first few firmware updates would be and they've stayed on schedule.  

The lack of a more locked down date does concern me.