SignalProcessingAndInformationTheory

This is a class for the Bachelor of Science in Bioinformatics. The course is in English, so this page is in that language

Schedule, location and links

End of course summary

We spent three months deepening the subject of study, and the available contents are summarized below

Topics slide video
Course Overview link 1
Signals and Systems link 1
Fourier series and signals space link 1, 2
Fourier transform and convolution link 1, 2, 3, 4
Sampling and digital signal processing link 1, 2, 3
Analog and numeric filters chapter 1, 2
Random signals, correlation, Wiener's theorem and signal statistics chapter 2
Information Theory chapter 2
Signal Processing over Graphs link 1, 2, 3, 4
Intermediate tests 1st, 2nd, 3rd

Progress of individual lessons

  • tue 8/3, fry 11/3 - Course Overview: After the illustration of the telematic tools of reference for the course, we introduce ourselves to each other. This is followed by a brief introduction on the nature of signals and their processing, in order to provide an overview of the objects we will deal with. In accordance with what I had proposed to myself, some applicative aspects of signal processing and information theory to biological, medical, genetic, microscopic and structural contexts are then outlined, up to mentioning the most recent applications. But, really, don't be afraid of this, I just wanted to arouse the curiosity to study signal processing, at least to understand everything you can't understand about more advanced topics yet!
  • fry 11/3, tue 15/3, fry 18/3 - Signals and Systems: This series of slides covers the contents of chapter 1 in the Signal Basics book (but ask me for the access credentials). After a brief introduction on signals (co)domains, on the Fourier analysis family and on the relationships between impulse response, convolution, frequency response and filtering, different classes of signals are defined, on the basis of their behaviour over time and asymptotic, and some notes are given on common operations performed on signals, their combination, and graphics. The exposition continues by illustrating the more commonly used signals, followed by a review of complex algebra, and the role of complex exponential in frequency analysis. Since signal processing is typically implemented as the transit of the signal itself through a linear operator, the characteristics of the latter are exposed, in contrast to those of non-linear processing units.
 - 
  • fry 18/3 - Fourier series and signals space - 1st part: After an introduction to phasors notation, the representation of periodic signals as an ordered set of Fourier coefficients is given, together with their reconstruction formula known as Fourier series. Then the alternative representations for real signals are given.
  • tue 22/3 - Fourier series and signals space - 2nd part: Exposition continues with the effects of using only a limited set of coefficients. This section closes with the proof of the Parseval’s theorem which gives a typical effect of the exponentials orthogonality property. The second part of this series of slides deals with the concepts of vector algebra when applied to signal spaces and, with the excuses of motivating the expression of the Fourier series, we take a guided tour through notions as the basis of representation of a vector space, the norm of a vector.
    • link to the video registration - Audio had a whistle and the classroom camera refused to work, so there is no blackboard visible. In the middle the classroom computer has hang, I had to reboot it. Next time I will bring a spare webcam from home, or maybe a tablet.
  • fry 25/3 - end of Signals space. Fourier transform and convolution begins, 1st part. After introduction to inner product spaces and Schwartz inequality, its application to the spaces of periodic, energy and power signals are shown by definition of the inner product formula for these spaces. The properties of signals are put in relation with definitions for metric spaces. Furthermore, its shown how any linear transformation is the result of an inner product, and that many signal processing results fall into this case. Finally, the concept of vector space is applied to linear operators, thus arriving to the notion of dual space. The extension of Fourier series analysis to non-periodic signals is then introduced, the Fourier transform defined as well as the antitransform, and the relationship in between Fourier series and the transform of a period. Concepts of Cross energy, Orthogonality and Schwartz inequality are given, as well Parseval’s theorem for energy signals
    • link to the 4th slide set: Fourier transform and convolution
    • link to the video registration - It was me who last time has failed to make the classroom cam to work, today it worked fine. The lesson is in Italian, as foreigners are no more following the course. Next lessons will be in Italian too.
  • tue 29/3 - Fourier transform and convolution - 2nd part: After the definition of Energy density, a first list of properties for the Fourier transform are proved, namely linearity and conjugate symmetry, duality, initial (or in the origin) value and area, time and frequency shift, conjugation and change of scale. Then the Dirac impulse is defined, along with the transform of a constant, and of periodic signals.
    • link to the video registration - Today you can finally ear the superb audio quality obtained by using my new radio microphone, I must remember to select it in the Zoom's settings, otherwise it uses the one from the pc. Oh well, the quality of a video, it's more than 50% the quality of its audio!
  • fry 1/4/ - Fourier transform and convolution - 3rd part: Sampling and sifting properties of the Dirac impulse, definition of Impulse response and of Convolution. Its commutative property, graphical construction and memory function. Convolution with a translated impulse. How convolution and multiplication are related. Frequency multiplication or filtering, Frequency response and its measure, Linear phase and delay, Cascaded systems, Transform of a triangle.
    • link to the video registration - Today I forgot my USB Webcam, so the one of my laptop was used, with a little worse image quality and worse screen perspective
  • tue 5/4 - First intermediate evaluation test - This should be done in presence, but we have been advised that the classroom will not be usable due to an antifire refurbishment.
  • fri 8/4 - Fourier transform and convolution - 4th part: Example of relationship between frequency and impulse response with the help of dfilter, intro to decibel (in the video), frequency convolution i.e. modulation and windowing. Spectral resolution and leakage. Short time frequency analysis. Fourier transform of derivated and integrated signals. Definition of a pulse train and its Fourier transform. Periodic signal transform, revisited.
    • link to the video registration - The last half an hour is missing, but for the rest quality is great, with a lot of practical examples
    • link to Octave code with which examples of execution are implemented as computer proof of the topic dealt with in theory. For details, read the Esercitazioni.pdf file. If someone can verify its compatibility with Matlab (TM) it will be fantastic, maybe it can be a means to make up for the lack of participation in the first evaluation test happy smiley
  • tue 12/4 - Sampling and digital signal processing -1st part: We have shown how a limited bandwidth signal can be fully described by knowing only its samples, if taken at a sufficiently high rate. Insights were given on the consequences of non-compliance with the requirements, as well as how to guarantee them. At last, implementation aspects such as A/D were addressed
    • link to the 5th slide set: Sampling and digital signal processing
    • link to the video registration - Today I found the cells of my radiomicrophone discharged, maybe I left it "on" last time? Hoping that after Easter we will be more numerous, today we was only two. I think that the possibility for you to pose questions is important!
  • fry 22/4 - Sampling and digital signal processing -2nd part: We have discussed the different types of Fourier analysis for the digital domain, such as DTFT for indefinite sequences, or DTF and its inverse in the temporally limited (or periodic) case. The description of the DTFT as calculated on a unitary circle opens the door to the definition of the zeta transform. Finally, a brief excursus on the topic of genomic signal processing has been started.
  • tue 26/5 - Sampling and digital signal processing -3rd part: we completed the discussion on the applications of DFT to genome and protein sequences, including how to interpret the DFT as a filter bank. Then, we begin to tackle the implementation of digital filters, starting with batch processing performed as a product in frequency, and highlighting the details on discrete and circular convolution, and the convolution between infinite sequences and a numerical impulse response
  • fry 29/4 - Analog and numeric filters 1st part: After having talked a lot about filters, some words about them! Characterization and types of analog filters, linear phase, digital filters, transversal filter or FIR, Analysis, Synthesis and Approximation. Numerical realization of the transversal filter. Moving average, High-pass or band-pass, First order FIR filter, Differentiator, Comb. First order (IIR) filter, Applications, EMA.
    • OPIS survey - Please fill it as described here (Italian) or there (English) and when (if) prompted enter the code XEZW9RBR
    • link to the Analog and numeric filters chapter - I no longer have time to prepare a set of slides, so from now on I will use the pages of the translated book directly
    • link to the video registration
    • we also ran the dist-fase.m Octave script, for the analysis of phase distortion
    • link to some video about spectroscopy (1) (2) (3) which is a form of first order FIR filter; see again pages 32-34 of the first slide set on Fourier Transform Infrared (FTIR) spectroscopy
  • tue 3/5 - Second intermediate evaluation test - This has been done in presence.
  • fry 6/5 - Analog and numeric filters - 2nd part: Numeric filters, FIR synthesis starting from the continuous time description, Zeta transform and filtering, Filters with finite and infinite impulse response, Architecture with a direct and canonical form. Random signals, correlation, Wiener's theorem and signal statistics - 1st part: Random variables, Probability density, Histogram, Expected value, moment and central moment, Multivariate random variable
  • 10/5 - Random signals, correlation, Wiener's theorem and signal statistics - 2nd part: Stationary and ergodic processes, Correlation, covariance and autocorrelation, Correlation of an ergodic stationary process, Autocorrelation and intercorrelation of deterministic signals, Properties, Wiener’s theorem
  • 13/5 - Random signals, correlation, Wiener's theorem and signal statistics, 3rd part: Autocorrelation function for a Periodic signal and a Band-limited white Gaussian process, Multidimensional Gaussian, Statistical independence for uncorrelated Gaussians r.v., Gaussian process. Spectral estimation, Periodogram and Autoregressive spectral estimate (hints). Pearson correlation coefficient. Information Theory, 1st part: Sorgente discreta senza memoria e sua entropia, sorgente binaria ed L-aria, Sorgente discreta con memoria e Markoviana, Entropia differenziale di sorgente continua e di sorgente gaussiana.
    • link to the Information Theory chapter - At the moment I haven't translated it yet, I will
    • Link to the video registration - Damn, after about two hours the cells of the wireless microphone are exhausted, so the last 1/3 (on information theory) is silent frowning smiley
    • passages of the demonstration that an incorrelated multidimensional Gaussian p.d.f. has statistically independent marginal r.v.
    • we also ran the filtra_gauss.m Octave script, showing that a filter witha gaussian input process has an output process which is gaussian too
  • 17/5 - Information Theory, 2nd part: Misure di informazione per una coppia di v.a.: Entropia congiunta e condizionata, Informazione mutua media. Equivocazione e noise entropy. Capacità di canale discreto. Application of information theory to biochemical signaling systems.
  • 20/5 - Signal Processing over Graphs, 1st part: these last lessons are held by Prof Stefania Sardellitti. Introduction to Graph Signal Processing (GSP), Basic tools from Graph Theory, Laplacian, eigenvalues, quadratic form.
    • Link to the video registration - This time I forgot the radio microphone closed until 31:33, before there was only the poor fixed microphone. But I managed to follow the teacher with the cam
    • Link to the slides
  • 24/5 - Signal Processing over Graphs, 2nd part (Prof. Sardellitti): Constant eingenvector for the null eingenvalue, multiplicity of the latter and connected graph components. Graph Signals: gene expression level, vehicular traffic intensity, electrocorticography signals. Graph signals on time and space domains. Spectral Graph Theory. Graph-shift operator, Graph Fourier Transform, a cyclic graph gives a DFT, Parseval equality, graph frequency domain. Graph Total Variation, eigenvalues as frequencies. Smooth signals defined on a graph are said to be band-limited, i.e. when its GFT ŝ is sparse.
  • 27/5 - Signal Processing over Graphs, 3rd part (Prof. Sardellitti): Graph Fourier basis for directed graphs, Graph Sampling Theory for bandlimited signals: vertex-limiting and band-limiting operators, Perfect localization conditions, recovery of a graph signal from a subset of verticies. Topology inference from data, application to Electrocorticography signals.
  • 31/5 - Signal Processing over Graphs, 4th part (Prof. Sardellitti): Graph Signal Filtering, FIR graph filter, Frequency response of FIR filters. Cancer subtype classification, Distinguishing power. Topological signal processing: simplicial complexes, cell complexes
  • 3/6 - Third intermediate evaluation test